Archiving Twitter

Let our journalists help you make sense of the noise: Subscribe to the Mother Jones Daily newsletter and get a recap of news that matters.

The Library of Congress plans to begin archiving all Twitter posts. Impressive! Except, not so much:

When do you start?
The agreement has been signed, but we still have a lot of technical details to work out — how we’ll technically transfer it, and when. There’s a built in six-month window, so we don’t have the live Twitter archive at any given time. There is a window for people if they want to delete their tweets, things like that.

There’s a built-in lag?
Yes, so once the transfer is complete, if a researcher comes here, we’ll let them know that it’s 2006 till six months prior. And there’ll be a rolling period of transfers after that.

How much will it cost?
Well, it’s a gift; we didn’t pay for it. But it will be the cost of storing what is, right now, around 5 terabytes, and the staff effort of maybe one full-time person over the years.

Five terabytes of storage? Seriously? That’ll set you back about a thousand bucks. Make it a fancy RAID array and maybe it’s a couple thousand. They needed a gift for this?

And I learned something else new: namely that (a) Twitter’s archives are remarkably small, and (b) they exist. I always figured there was no good way to search Twitter because they only kept tweets for a certain length of time. But no. They’ve got ’em all, and the database is so small that it could be indexed in a few hours. So why is searching Twitter so hard? And will researchers really have to “come here” to search the archives? That was left unclear at the end of the interview, but it sounds like this is Twitter’s call. Putting it online sure sounds like a better idea to me.

SIX TRUTHS

Reclaiming power from those who abuse it often starts with telling the truth. And in "This Is How Authoritarians Get Defeated," MoJo's Monika Bauerlein unpacks six truths to remember during the homestretch of an election where democracy, truth, and decency are on the line.

Truth #1: The chaos is the point.

Truth #2: Team Reality is bigger than it seems.

Truth #3: Facebook owns this.

Truth #4: When we go to work, we're in the fight.

Truth #5: It's about minority rule.

Truth #6: The only thing that can save us is…us.

Please take a moment to see how all these truths add up, because what happens in the weeks and months ahead will reverberate for at least a generation and we better be prepared.

And if you think journalism like Mother Jones'—that calls it like it is, that will never acquiesce to power, that looks where others don't—can help guide us through this historic, high-stakes moment, and you're able to right now, please help us reach our $350,000 goal by October 31 with a donation today. It's all hands on deck for democracy.

payment methods

SIX TRUTHS

Reclaiming power from those who abuse it often starts with telling the truth. And in "This Is How Authoritarians Get Defeated," MoJo's Monika Bauerlein unpacks six truths to remember during the homestretch of an election where democracy, truth, and decency are on the line.

Truth #1: The chaos is the point.

Truth #2: Team Reality is bigger than it seems.

Truth #3: Facebook owns this.

Truth #4: When we go to work, we're in the fight.

Truth #5: It's about minority rule.

Truth #6: The only thing that can save us is…us.

Please take a moment to see how all these truths add up, because what happens in the weeks and months ahead will reverberate for at least a generation and we better be prepared.

And if you think journalism like Mother Jones'—that calls it like it is, that will never acquiesce to power, that looks where others don't—can help guide us through this historic, high-stakes moment, and you're able to right now, please help us reach our $350,000 goal by October 31 with a donation today. It's all hands on deck for democracy.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate