VaryByQueryString

Under caching section (standard) there are checkbox to control caching behavior for an item. In my case it was the layout. What I was seeing is that the same detail page, when passed different item id in the query, was displaying the same (first) item. My VaryByQueryStirng was unchched but Cachable was checked, therefore it was caching the first item.