If you have any questions or concerns about the deprecation schedule outlined in this blog post, Deprecations - Spring 2012, and Working with Timelines please detail them below.
Thanks!
If you have any questions or concerns about the deprecation schedule outlined in this blog post, Deprecations - Spring 2012, and Working with Timelines please detail them below.
Thanks!
Replies
Hi Taylor. What will happen if you keep the 'page' parameters in your call? Will it just return the first set of tweets?
yeah thats what happened to me
Is there a test environment available or some test param we can specify to the production change?
I have no problems with removing the deprecated methods and simplifying other calls to always include entities and RTs. Makes sense. But I'm surprised to hear that "page" is going away. I don't remember seeing any guidance on this before today.
Removing the page parameter will break my app, and it seems like it will break many others. I think May is a little too aggressive for something that has been such a core part of the API from the beginning, and which even right now is listed to work perfectly fine in the official documentation.
I only have one concern with the announcement, and that's the deprecation of the public timeline as part of the REST API, instead making users go to the Streaming API.
In teaching programming to friends, colleagues, and kids, the Twitter API is one of the most interesting and accessible ways to get them started. The sense of immediacy and "real people"-ness is extremely valuable when you're trying to get someone excited about a skill that can feel arcane and comprised of minutiae.
The public timeline has been my go-to (though not always for audiences of children), and moving it into the Streaming API means it will no longer be practical to use it during lessons. While I could maybe still use some client libraries to mask the complexity in some cases, JSONView and Firebug are no longer options.
Twitter is also limiting its utility as an example in the documentation of lots of code utilities and libraries that work with HTTP and REST. I hope you'll consider keeping the public timeline available in the REST API, even if it's with heavier caching or stronger caveats on the quality and immediacy of the data.
"All methods currently supporting cursoring or usage of since_id and max_id will remove support for "classic" pagination through the "page" parameters."
Will supplying a sufficiently large since_id and count return all of the tweets we have would been
able to access by paging previously? The count parameter is currently capped at 200 - will this be increased to 3,200?
In other words, will:
/1/statuses/user_timeline.json?count=3200(no since_id supplied)return up to 3,200 statuses (minus deleted tweets) that we would have been able to access previously using
Thanks!
Hi David,
We still have limits to how many tweets you can request in a single request -- it takes processing power for us to assemble timelines and the maximums are in place to increase the chance that assembly can happen in an acceptable amount of time.
For a completely vanilla new user to your system, you would request a user_timeline with no since_id or count_id:
/1/statuses/user_timeline.json?count=200&include_rts=trueI would then store both the most recent tweet ID in that response as well as the oldest you retrieved. In the future, whenever you wanted to get more recent tweets by the user, you would use the most recent tweet ID as the since_id value, then work yourself up forward in time through coordinating since_id and max_id.
To work through the past of a user_timeline, you would take the oldest tweet ID from that first request and use that tweet ID as the max_id value (or subtract it by one to help prevent duplication). This would yield you the next oldest 200 tweets since the one specified by max_id. You could then continue this process down until you've worked your way through the set of 3,200. Working with Timelines explains this much better.
Hi Taylor,
Thanks for the response. This suggested approach using the min_id from each successive batch of 200 tweets as the max_id for the next batch definitely creates less confusion and is overall a cleaner approach than paging. However, it also limits our ability to fetch the user's tweets in parallel. Currently we process tweet pages in parallel, but once the page parameter becomes deprecated, we'd have to wait for the response back from twitter before fetching the next batch of 200 tweets. Not sure if this affects anyone else, so it may not be a huge deal. We could definitely work around this, but just bringing it up in case it affects a lot of other developers.
Best,
David
Does the page parameter affect the SEARCH method as well?
No, the search API is untouched by any of the changes in this announcement.
I've got problem with the page parameter on the search API.
The following request works well:
but this second one gives "Invalid Query" error, but I cannot understand why:
Thanks
It looks like you're ending the query with a boolean operator "OR" -- make sure you trim off extra operators if you have no argument fo rhtem.
...other things:
1. The depreciation of the statuses/public_timeline for site streams. I just want to say not all clients can use site streams. SSL as a requirement for site streams as one of the reasons.
2. The depreciation of statuses/friends and statuses/followers. The problem with GET followers/ids and POST users/lookup (at last check) is that it doesn't return friends/followers in the order they followed/were followed as statuses/friends and statuses/followers do.
I think Twitter needs to address the second point because it is a loss of functionality.
If followers/id and friends/id do not return the followers and friends in the order the follow/following happened then how exactly is the Twitter web interface able to display followers and friends in that order?
In regards to "OAuth 1.0A [10] operations must be performed on the api subdomain. Older http://twitter.com/oauth/* paths will cease to be functional."
Reading the docs for authentication, everything would still seem to be using this "older" method.
The docs say to use "https://twitter.com/oauth/request_token" not "https://api.twitter.com/1/request_token".
I just want to make sure the request_token, access_token, authorize and authenticate are reamining untouched...
Thanks
Which docs on dev.twitter.com have you found that still reference this path?
Hello Justin! did you get an answer about your question? I am using now the https://api.twitter.com/1/request_token, authorize, ... are these the corrects urls?
If you're using https://api.twitter.com/oauth/request_token, https://api.twitter.com/oauth/access_token, https://api.twitter.com/oauth/authorize or https://api.twitter.com/oauth/authenticate you are using the correct paths.
So, just for clarification, despite "All non-versioned API endpoints ... will cease functioning" OAuth remains non-versioned?
How are we supposed to get all the favorites without the page parameter? Favorites supports only since_id and page parameter (https://dev.twitter.com/docs/api/1/get/favorites).
Edit: The doc has been fixed, we can use max_id, thanks :)
Getting rid of the page parameter for timelines and mentions will greatly reduce the speed of our app which analyzes your historical tweets. Right now we can ask for 16 pages of the user timeline at once and then assemble results once. Removing paging means that our app will be 16x slower than it is now because we will have to make requests synchronously.
Is there anyway for us to avoid the 16x decrease in performance caused by deprecating paging?
Such batching will cause you to lose Tweets eventually. On a stopped timeline this approach may work, but there's no way to estimate the "slowness" of a timeline at a given point in time.
Imagine a case where you request pages 1 and 2 simultaneously. If page 2 returns results before page 1, and in the (very small) time between these responses a new Tweet is added to the timeline, then the Tweet which was originally at the bottom of page 1 will never have been received by your application.
We have a similar use case as Eli in that we'd like to process /user_timeline quickly to pull historical tweets when a user first signs up for our service. We rely on site streams thereafter. Since a user is unlikely to send out a new tweet while they are our signing up for our service, it seems relatively safe to use this approach without missing tweets. We also immediately add the user to a site stream and can detect if they do send out a tweet and push the pages to be off by one.
Definitely will take this into consideration.
Walking a user timeline is among the most expensive things you can do in the API -- to be most polite to the API, we would ask that you do this in tandem rather than in parellel anyway. It's additionally more expensive to us for you to crawl a timeline using page and count instead of since_id and max_id and to do so in multiple simultaneous requests is stressful.
Sometimes your users should just have to wait.
I just wanted to re-iterate that, like David Tran, we only need to perform this parallel operation once when a user signs up. After that, we can be extremely polite to the API -- in fact we currently only call the API about once per day per user after that.
Perhaps an alternative would be to allow applications to query the API with a large count parameter, but only a handful of times per hour? Or to make requests for large counts count as some multiple of smaller requests?
I appreciate and understand your concerns around over taxing the API. But, as I'm sure you've experienced on other sites, the shorter time we make our users wait for results, the more likely they will be to stick around.
Regards,
Eli
Hi Taylor.
With the page parameter missing, I might have to resort to walking a user's timeline. If there is another way, please le me know. I'll explain:
I run TwimeMachine (http://www.twimemachine.com). Users can go to an arbitrary position in their timeline based on a page number. The max is 128 pages. If they enter 40, it retrieves the tweets from there. How will I able to get to an arbitrary by only using since_id and max_id?
The 'page' paramater allows me fetch a set of tweets in the middle. The only way I'll be able to do this now, is if I traverse/walk to that point, which will be needlessly expensive.
Simon,
In your case, I'd recommend not offering that feature any longer. There are a few things you'll be able to do to support it, by learning more about how Snowflake IDs are generated keeping a rough timeline/mapping of days to Snowflake ID ranges then using those as since_id and max_ids -- but that seems like a lot of overhead.
Pagination on timelines is an illusion we've helped perpetuate until now. If it's an illusion you'd like to continue supporting, you're probably best off doing so against your own collection of tweets.
Long-term, you should be using either the Streaming API's follow filter or Site Streams to track the tweets of existing users in real time to populate your data set. When a new user comes and you want to provide them with their 3,2000 tweets, you would walk their user timeline politely and synchronously in a handful of requests.
Thanks Taylor.
I've recapped the API changes. Will it in anyway be possible to actually retrieve the oldest tweets without traversing all the way down? It doesn't look like it?
It's possible but so tough it's probably not worth your time -- you'd need to do some mapping between dates and snowflake IDs (and for pre-snowflake IDs..) then make requests to the user_timeline with ID boundaries near the created_at date for the user's account as the since_id and max_ids pointing to some reasonable point of time after that created_at date.
I've been away awhile and I return totally baffled ! So much jargon ! All I want to do is to post a tweet and possibly evoke some kind of response
It isn't clear from the blog post and this discussion if atom results will still be available via the search api. We're currently running queries against the api using urls like "http://search.twitter.com/search.atom?q=". Will these searches continue to work or will we have to transition to json?
Thanks!
The Search API has no changes in this announcement. ATOM results in the Search API will not be effected. I still recommend transitioning to JSON.
We've been planning a transition to JSON for a while now, but May was earlier than we wanted to do so. Thank you for the clarification.
Is it possible to have a callback for this new user timeline? I used to call a twitterCallback2 function from this resource http://twitter.com/javascripts/blogger.js
Thank you
Hi,
we have integrated the twitter api on our wine & food social network Vinix. It worked very fine since 3 days ago when we've started to experience some performance issues with the twitter api. We've noticed that opening a page of the network where we use the twitter api takes a very longer time than a page where we use not (10-12 seconds more than before) while since 3 days ago all was pretty fast.
We have the call to the api in our common.php file that is read before any other element of the page, so, now we have the whole website (almost all), very slowed down because of these api performances issues.
The major part of the integration works when you're registered and logged in but now (since the last 3 days) you can see the performance difference opening for example this page where we don't use the api:
http://www.vinix.com
and this page where we do use the twitter api:
http://www.vinix.com/detail.php?ID=25016 (look to the bottom to see the integration).
Since 3 days ago, this second page opened in less than 1 second. Now it takes more than 20 and we're strongly convinced that this is not on our side because if we remove the use of the api, we back immediately to a very fast response of the page.
All of our users using the twitter api are experiencing the same performance issue and it is very frustrating for us. Looking to the twitter api status page, I can see that in the last 4-5 days it was always under the top performance (tellow rounded button) but we've this integration runing since november 2011 and this is the first time we have this kind of "slowing down" issue so I think this might be something related to deprecations.
Any help?
Cheers, Filippo Ronco
Italy
Little update, may help someone: we have changed the call to the api from xml to json and all is back very fine.
Cheers, Fil.
Hi All
This is Vijay . I have the following question on API house keeping.
{{{}}{}{}}Previously I used above Paging code to get the tweets from twitter API. Just I am dividing the NumberOfStatuses by Count per page and looping through the each page and adding the response to list. If I use following code I am getting the tweets count as 3200 for sample account ( Passing .SinceStatusId =0).
When I Use the following new code which is implemented based on working-with-timelines (https://dev.twitter.com/docs/working-with-timelines) docuemnt, i am getting the tweets count as 3400. Some difference is coming between old paging code and new code . This difference is coming in Reweets and mentions also.
When I use this code for other accounts like which has less no of tweets, retweets and mentions it is working fine.
Based on working-with-timelines concept I removed the paging concept and implemented the following logic. each time I am passing the Max Tweet Id for the next request and processing the records until tweets count comes to Zero.But If i do like this I am getting different Tweets count.
{{}}}{//Iterate through the collection to insert each and every tweet in the database.{}}else{}{}}What is the difference between these two codes and which one is correct. Please explain clearly.This problem is coming for Tweets and Mentions also.
Thank you
Still If we use the paging logic what will happen? In the API document paging is mentioned as optional. Still API will suport paging?
According to the blog post, the page parameter will not work from May 14.
Is there any way to have the API return a range of tweets immediately following a certain tweet ID? If you specify the since_id parameter, the results always start from the very latest tweets so you may not get all the tweets until since_id.
The timeline length limitations will still be present in that a home timeline, mentions timeline, and user timeline are finite. The since_id parameter does what you're asking about, and as long as the number of tweets in that timeline does not exceed that timeline's maximum since that since_id, you should be able to retrieve them all (same as is possible today).
What if I want to retrieve only the next 200 tweets after a certain tweet ID? In other words, I'd like to call the API with the parameters "since_id=ID&count=200". Unfortunately, this slice is from the top of the timeline downwards, not the block of tweets above since_id.
Of course, I could just omit the count parameter, but it's a waste to retrieve everything and then throw away all but 200 tweets.
Hi
This my first post in Twitter developers site. I am quite new to API twitter. I am running website http://bit.ly/HszKpJ with 5 twitter feed sections/widgets. Average is 5 tweets a minute. I learner that twitter feed is limited to 150 tweets and hour. As soon as i reached that limit my widgets turn blank and they reset in following hour. I am hoping if there is anybody who can help me out with this issue
Cheers
What if those don't update as often? You know the rate limit per hour, so divide that by 5 and that's the number of times each section can update per hour. Budget your API calls.
Hi
I am currently using the GET statuses/user_timeline method via https://api.twitter.com/1/statuses/user_timeline.xml?. From the documentation, I don't quite understand if this method will be deprecated on May 14th? Or is it just the previous un-versioned endpoint http://twitter.com/statuses/user_timeline.xml that will then need to be accessed as I have been doing? Thanks.
You are correct -- the old form is being deprecated. But there are also some particulars about how you interact with the user timeline that are changing on May 14th -- namely that you'll no longer be able to use the "page" paramater for pagination and need to use the methods suggested in Working with Timelines.
I was using GET statuses/public_timeline to get the length of the latest status ID. Now that this API is going away, would it be possible to add status_id_length to GET help/configuration?
What's the use case for knowing the specific length? It's a 64 bit unsigned integer that is safest when consumed as a string. Is what it currently is at more noteworthy than where its upper bound is? Interested to hear the use case, thanks.
My specific use-case is outputting a Twitter status ID in the column of TTY: https://github.com/sferik/t/blob/dd41bfdb12b8a50ede4059a00a3b482a88a6bb38/lib/t/stream.rb#L13
Right now, the length is hardcoded to 18, but that code will break once status IDs cross 1 quintillion. :)
Hi everyone,
Thank you for your continued efforts in preparing for the upcoming deprecations. Based on your feedback, we've decided to more incrementally phase these deprecations over the next several weeks. Stay tuned next week for an updated deprecation schedule. None of the currently scheduled changes will take effect on May 14th, 2012 but we still encourage you to continue implementing the necessary changes in your applications.
Thanks!