Library of Congress digs into 170 billion tweets

January 9, 2013
By

An effort by the Library of Congress to archive Twitter posts has amassed more than 170 billion tweets, which the library is now seeking to make available to researchers and other interested parties.

Created in 1800, the Library of Congress serves as the unofficial library of the United States, as well as being Congress’ official research library.

In April 2010, the library signed an agreement with Twitter to gain access to all public tweets since the site’s founding in 2006.

“Twitter is a new kind of collection for the Library of Congress but an important one to its mission,” Gayle Osterberg, the library’s director of communications, wrote in a blog post. “As society turns to social media as a primary method of communication and creative expression, social media is supplementing, and in some cases supplanting, letters, journals, serial publications and other sources routinely collected by research libraries.”

Osterberg wrote that the library has completed digitally archiving all of the tweets it currently possesses and is now working on how to best make them available to the public. The library already has received about 400 requests from researchers all over the world looking into topics ranging from the rise of citizen journalism to tracking vaccination rates to predicting stock market activity.

The archive promises to keep growing fast. Currently, the library is processing roughly 500 million tweets per day, up from about 140 million daily messages in 2011, according to the blog post.

Making such a vast database publicly available is proving to be a challenge unto itself, according to the Library of Congress.

“It is clear that technology to allow for scholarship access to large data sets is lagging behind technology for creating and distributing such data,” library executives wrote last week in a government white paper updating their progress. “Even the private sector has not yet implemented cost-effective commercial solutions because of the complexity and resource requirements of such a task.”

Currently, the library is working on partnerships with the private sector that would at least allow access to the archives in its Washington reading rooms.

The Twitter archive might be its biggest and most challenging effort, but it’s not the first time the Library of Congress has sought to document the digital world.

Since 2000, the library has been collecting pages from websites that document government information and activity. Today, that archive is more than 300 terabytes in size and represents tens of thousands of different sites. The library’s entire collection of printed books has been estimated to total about 10 terabytes of data (although staff at the library suspect it’s probably more).

 

Read more: CNN

Tags: , , , , , ,

Comments are closed.

Follow The Feminist Wire

Arts & Culture

  • Poem Suite: Silencing RasiqraRevulva-_3-4_Profile_;__Romancing_The_Own_;__Waning_Gibbou-glitch_headshot

    Elie Wiesel writes, “I swore never to be silent whenever and wherever human beings endure suffering and humiliation. We must always take sides. Neutrality helps the oppressor, never the victim. Silence encourages the tormentor, never the tormented.” Silence binds perpetrators and victims, creates a confederacy of secrets. Here, these women [...]

  • 3 poems by Sarah Kortemeier SarahKortemeier-_Baby_Fever____Stone_with_Nineteen_Corners____The_-sarah_kortemeier_outdoor_color_by_jennifer_mcstotts

    The Mountain   The mountain is really a series of itself. Deeper pockets of sky color float in its canyons. In certain seasons, it’s difficult to tell rock face from snowfall. The ridge line looks much sharper than it must, in actuality, be. When you climb, the summit is sometimes [...]

  • 3 poems by Arielle Greenberg Wormwood portrait LA

    Who I’d Like to Meet   I am on tiptoe scanning our tallest bookshelves for something to pack to read on the plane.  I am scanful, tippy-toed: a girl without boots.  I am shorty.  I want to read something great, as in literary, and beautiful and daring, and something hobnailed [...]