Hi Taylor, what causes a tweet to become unavailable for searching? I don't see any reference to a date range limitation in the API ref doc for search: https://dev.twitter.com/docs/api/1/get/search
The index varies in length over time, but generally only goes back about 1 week. Other factors than age are also at work, as the Search API does not index all public tweets. See this FAQ more info:
Twitter's search is optimized to serve relevant tweets to end-users in response to direct, non-recurring queries such as #hashtags, URLs, domains, and keywords. The Search API (which also powers Twitter's search widget) is an interface to this search engine. Our search service is not meant to be an exhaustive archive of public tweets and not all tweets are indexed or returned. Some results are refined to better combat spam and increase relevance. Due to capacity constraints, the index currently only covers about a week's worth of tweets.
The user timeline API is the definitive source of tweets by a specific author. For a collection of recent Tweets by more than one user, consider creating a Twitter List and leveraging the list timeline. The Streaming API is often the best-fit choice when seeking completeness.
If the tweets you're looking for aren't available in search and you think they should be, consult this support topic for further instructions to contact @Support.
So if I understand correctly, one can retrieve all of a certain user's tweets, using timeline, by paginating through results with a max of 3200. But when looking for tweets by any user that contain a certain hashtag, because search is used, results go back give or take a week or 1500 results, whichever comes first.
Replies
They aren't deleted, they're not available for searching. Some sites provide some historical views of tweets -- try topsy.com
Hi Taylor, what causes a tweet to become unavailable for searching? I don't see any reference to a date range limitation in the API ref doc for search: https://dev.twitter.com/docs/api/1/get/search
Thanks
The index varies in length over time, but generally only goes back about 1 week. Other factors than age are also at work, as the Search API does not index all public tweets. See this FAQ more info:
Thanks Taylor.
So if I understand correctly, one can retrieve all of a certain user's tweets, using timeline, by paginating through results with a max of 3200. But when looking for tweets by any user that contain a certain hashtag, because search is used, results go back give or take a week or 1500 results, whichever comes first.
Hi Andy,
More or less that is the state of things. You may be able to obtain the historical data you're looking for through Topsy.com or DataSift and Gnip.