Matt Cutts recently released a video answer to the question, “Is page speed a more important ranking factor for mobile sites?”
In his reply Matt explains that site speed, when used as a ranking factor, will be considered contextually—not in an absolute number—because website speed varies based on country, bandwidth, and a variety of other factors. He encourages website owners to look at the “neighborhood of websites” returned in search results with your own and compare speed, cautioning that if you’re the outlier, you very well could see your rankings affected.
In the video, Matt explains that page speed on mobile is not considered more important by Google though he says:
. . . If you’re using your mobile phone, you care a lot about whether it will load in a reasonable period of time and so we’ll continue to look at ways to improve the way that we find out how fast a site is, the page speed for a particular page, and then try to figure out whether it makes sense . . . okay, if we want users to be less frustrated, then maybe over time it does make sense to incorporate more into our rankings, or more for mobile . . . something along those lines.