Tuesday, May 8, 2012

Pagination with Spring MVC, Spring Data and Java Config

Spring 3.1 has a lot of features for limiting the boiler-plate code you have to write for common functionality.  One great example is pagination.  Getting paged data from a database and presenting it to the user is one of those tasks that everyone seems to reinvent even though it's common functionality that is never specific to your business.  Spring Data provides some facilities to add pagination to your application with a minimal amount of code.  The documentation on this is pretty good and it includes some steps on how to set up both in the data tier and the web tier.  Unfortunately, the documentation is unclear in a few places (see below) and doesn't explain at all how to use a Java Config.  This post tries to fill those gaps.


Data Tier
The first thing we need is a data tier that can return paged data from a database.   Spring Data provides a way to access the database without writing any code (almost). Instead of having to define and implement common CRUD methods you simply define an interface. You don't have to write the implementation because Spring will create a proxy that automagically fulfills your interface.  You just wire the interface and Spring does the rest.  I'm using JPA and here's a real example:

package com.fawnanddoug.gmailsyncer.repository;

import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.jpa.repository.JpaRepository;

import com.fawnanddoug.gmailsyncer.domain.SyncJobDetail;

public interface SyncJobDetailRepository extends JpaRepository<SyncJobDetail, Long> {
 
 public Page<SyncJobDetail> findBySyncJobIdOrderByLastUpdateDesc(long syncJobId, Pageable p);
  
}
Most of the work occurs on line 9 where I extend the JpaRepository interface.  Simply by extending that interface I get common CRUD methods like save(), findAll(), delete(), etc.  I also give my Entity Type (SyncJobDetail) and the key (Long) so that I get a strongly typed interface.  Now I have a repository that can create and modify Objects in my domain and all I did was extend an Interface.

I get some additional help on line 11.  There I define a method to grab the SyncJobDetails by their parent's Id and I sort them by the last updated time.  Spring can automatically infer what the implementation should look like based on the method name (which is admittedly ugly) and once again I don't have to write any implementation.  The first parameter will map to the SyncJobId in the query and the best part is it automatically adds support for Paging just by adding the Pageable parameter.  Pageable has methods like "getPageNumber" and "getPageSize" so I can easily specify that I want pages with 20 items per page and give me page number 7. The only trick here is the configuration which I'll describe below.

Web Tier
Now that we can pull the data from the database we need some way to serve it over the web.  This also requires a minimal amount of code.  Here's an example:

@Controller
public class HomeController {

 private final SyncJobDetailRepository repo;

 @Inject
 public HomeController(SyncJobDetailRepository repo) {
  this.repo = repo;
 }

 @RequestMapping(value = "/history/detail/{id}", method = RequestMethod.GET)
 public String historyDetail(Model model, @PathVariable long id, Pageable p) {
  Page<SyncJobDetail> page = this.repo.findBySyncJobIdOrderByLastUpdateDesc(id, p);
  model.addAttribute("page", page);

  return "historyDetail";
 }

Injecting the repository (line 6) and using the request mapping (line 11) are pretty basic Spring MVC concepts.  In short, if the user requests a page like /history/detail/123 then the historyDetail() method will get automatically invoked.  The real magic is on line 12 where the Pageable parameter is automatically inferred from the parameters that are passed into the request. This also requires some configuration magic which is described below.  Once we have the Pageable we can just pass it on to our repository to get the appropriate data.  The returned Page is then exposed to the View so that it can be displayed to the user.  The page has useful information like which page we're looking at, how many total items exist and which items were returned in this page.  Once again we have barely any code written and we get Paging essentially for free.

The View
Now that the Page Object is exposed to the view we can use that Page to display useful information to the user.  Here's an example JSP:
There's nothing particularly special here.  I created a pagination util tag lib since I reuse pagination in a few places.  It shows things like the total number of pages, the current page and how many elements are on the page.  As you can see all of this information comes from the Page object on lines 1 and 2.  The only other interesting item is that the content is exposed from the Page object at line 14.  Pretty straight forward and I didn't have to manually calculate or even query any of this information.

The only thing left is to post back to the controller when we want a new page.  In this case we just have to specify the right parameters for the page size and the requested page and we're done.  Here's an example of how I generate the URL for my "Next" button:
In line 1 I use "" as the url value so that we post back to the same page. I then add the "page.page" parameter for the next page (current page plus one) and use the same size that we're currently using. Nothing particularly special here and all of the values come from the Page object that was returned from the data tier.

The Gotchas
The code, as you can see above, is pretty minimal and can apply generically to any domain.  There are, however, a number of gotchas.  Let's explore these gotchas by working backwards from the View to the Web Tier to the Data Tier.

View Gotchas
The first one is related to the documentation.  In the above example I specify the next page parameter with the key "page.page".  The gotcha is that the docs say the parameter name is just "page" so it took me a little while to catch.  Technically you can specify any prefix you want (so it could be "foo.page"), but the important bit is that you must specify a prefix and then the parameter name.  The prefix can be configured on the PageableArgumentResolver which is described below.

The next gotcha is that the Java Doc for PageRequest (the Pageable implementation) says that it is 0-based but the Java Doc for Page says that it is 1-based.  In other words, when you request data the first page is 0 but when it returns data the first page is 1.  This can be pretty confusing and may lead you to try to compensate in your code for potential one-off issues.  In fact, I think it's intentional and the goal is that when you display data it uses numbers that a User wants to see (e.g. page 1 is "1") but still plays nice with databases that think of the first page as 0.  The PageableArgumentResolver (described below) will automatically handle the conversion which means you won't need code to compensate for this, but it can still be confusing.  In short, it does the right thing but may be hard to wrap your head around.

Web Tier Gotchas
I mentioned above that to get the Pageable parameter to work in your controller you need some configuration magic.  In order for Spring to know how to convert the parameter to a Pageable object you need to configure a HandlerMethodArgumentResolver.  Spring Data provides a PageableArgumentResolver but it uses the old ArgumentResolver interface instead of the new (Spring 3.1) HandlerMethodArgumentResolver interface.  The XML config can handle this discrepancy for us, but since we're using Java Config we have to do it a little more manually. Luckily this can be easily resolved if you know the right magic incantation:

@Configuration
@EnableWebMvc
public class WebMvcConfig extends WebMvcConfigurerAdapter {
        
        @Override
        public void addArgumentResolvers(
                        List<HandlerMethodArgumentResolver> argumentResolvers) {
                
                PageableArgumentResolver resolver = new PageableArgumentResolver();
                resolver.setFallbackPagable(new PageRequest(1, 10));
                
                argumentResolvers.add(new ServletWebArgumentResolverAdapter(resolver));
        }

The above is a standard Spring MVC Java Config.  It extends WebMvcConfigurerAdapter so that it can override some common methods and add functionality.  In this case we are overriding addArgumentResolvers so that we can add our PageableArgumentResolver.  The key here is that on line 12 we wrap the PageableArgumentResolver in a ServletWebArgumentResolverAdapater.  This lets us reuse the PageableArgumentResolver and still wrap it in the new Interface.  Now we get the Pageable object injected into our methods for free.

One other note is that I set the FallBackPageable to use Page 1 and a Page Size of 10.  This means that if there is no Page parameters set it will use Page 1 and a Page Size of 10.  However, if you were paying attention above you would note that Pageable is supposed to be 0-based.  The magic here is that the PageableArgumentResolver tries to automatically compensate for 0-based and 1-based issues so it will treat both "0" and "1" as the first page.  "2" is always the second page.  This can be a little disconcerting at first, but as long as your view is passing in the values that the user wants (e.g. page "1" is "1") then you'll be fine.

Data Tier Gotchas
I said above that this was about using Java Config but the sad news is that you can't use the JPA Data Repository scanning without XML.  A typical approach is to import the XML using something like this:
@Configuration
@ImportResource("classpath:com/fawnanddoug/gmailsyncer/config/jpa.xml")
public class MainConfig {

// beans go here

} 


Then you can have a simple xml config that looks something like this:

In the above, line 11 searches the specified package to find our repository interfaces.  It then generates the appropriate implementations. Now we can wire our repository into the Java Config and use the cool new features.  In our case the SyncJobDetailRepository will get autowired into the HomeController so we don't have to add any additional configuration.

Conclusion
And that's it.  With a minimal amount of code you can add paging to your application and still use strongly typed methods and services.  Just make sure to configure the appropriate parts and look out for 0-based / 1-based issues. 


13 comments:

  1. Thank you, its a great article and thanks for the gotchas. Is it possible to do paging with plain spring-jdbc repositories as mentioned in this article by using Pageable interface. My constraint here is i cannot use JPA as my project uses plain spring-jdbc.

    ReplyDelete
  2. Thanks! I think you can use the repository functionality with regular JDBC. Check out the spring data commons: http://static.springsource.org/spring-data/data-commons/docs/current/reference/html/#repositories

    ReplyDelete
  3. Great article! I like the fact that the Pageable class structure can be used without any of the repositories. I just have to mimic the spring data repository stuff in my own repository class structure.

    ReplyDelete
  4. I think its funny that you say "when you request data the first page is 0 but when it returns data the first page is 1", because the behavior that I observe is exactly the opposite.

    My responses are 0-based, so the first page number I receive from the server is 0. When I request page 1, I again receive page 0. Only when I request page 2, I will get back page 1. So it looks like the requests are 1-based and the results are 0-based. Confusing. At least now I know whats going on.

    Anyway thanks for sharing!

    ReplyDelete
  5. Great article, thanks a lot. How are you handling the totalElements and totalPages properties? These aren't part of the standard "Pageable" interface and I'd be curious to know how you're getting those and passing them around in your util library.

    ReplyDelete
  6. Hi Alex,

    Thanks! I use the Page object to get the totalElements and totalPage properties. The Pageable is the object you pass to the JPARepository to specify things like sorting direction. The Page is the object you get back from the JPARepository.

    Thanks Again!

    ReplyDelete
  7. Hi,

    We were recently working with pageable object in spring data(using hibernate) and found out that, the way spring data implements pageable is it loads all the data into hibernate entities and do a pageable on that. If you have a big chunk of data then it it was taking almost 2 minutes to get a page of particular entities. For us spring data flipped after the table size is more than 50000 rows. We anyway had to write our own query for that. So be careful if you want to handle big databases.

    ReplyDelete
    Replies
    1. Hi Nitin,
      Can you please share how you did that

      Thanks

      Delete
    2. How did you figure out that's what they are doing? I find it hard to believe they would implement it like that, I'm guessing there was something that you were doing wrong in your code or you are using some DB that is not fully supported?!

      Delete
  8. Thank you Doug, you saved me lots of time. I used your configuration code in my blog application, and it works perfectly! See my post at http://www.jiwhiz.com/post/2013/2/Implement_Bootstrap_Pagination_With_SpringData_And_Thymeleaf

    ReplyDelete
  9. Thanks for the writeup - the @Override addArgumentResolvers() method was tripping me up.

    ReplyDelete
  10. Hi
    How to download the this sample application

    ReplyDelete
    Replies
    1. There's no sample app to download. Just the code in the page.

      Delete