Lazy, Filtered and Sorted Vaadin Grid using External Data Source

In large enterprise application development projects, it’s usual to code use cases that, as the main requirement, define to show the user huge amounts of data in a tabular way. For this kind of scenarios, there is a particular web component that comes really handy: a Grid. Given that we like Vaadin’s framework, we are going to describe how to configure it in such a way that doesn’t restrict you about performance issues when doing that.

This post assumes that you have some knowledge of web application development using the Vaadin Framework and the Java Platform. But we’ll try to explain everything as much as possible. The version that we used to write this application is Vaadin 13, but this part of the API is almost the same since Vaadin 8, so it should be possible to use the same approach in older versions.

Setting up a Vaadin Grid, is really easy, just create an instance and then customize the columns, or even skip that part if your Pojo is sufficiently self-descriptive.

As an example, let’s assume that we want to show a grid with data for a person, which we can define as a simplified entity like this:

public class Person {



 private String name;

 private String lastName;

 private Integer age;

 

 public Person(String name, String lastName, Integer age) {

  super();

  this.name = name;

  this.lastName = lastName;

  this.age = age;

 }

 public String getName() {

  return name;

 }

 public void setName(String name) {

  this.name = name;

 }

 public String getLastName() {

  return lastName;

 }

 public void setLastName(String lastName) {

  this.lastName = lastName;

 }

 public Integer getAge() {

  return age;

 }

 public void setAge(Integer age) {

  this.age = age;

 }

}

And then we have a service that will have the responsibility of retrieving this information from a backend, like a database or a remote API. Let’s call it PersonService.

Suppose we want to show those entities in a Vaadin Grid, we would do this:

     List<Person> persons = getPersonService().fetchPersons();

     Grid<Person> personsGrid = new Grid<>(Person.class);

     personsGrid.setItems(persons);

This is fine for small amounts of data, but when dealing with large data sets some features, such as lazy loading, filtering and sorting, must be left to the backend, implementing that all at once can be a complex task, so let’s review them.

First let’s define what we are talking about when we use this term. Usually when we are handling a list of items, we can choose to obtain  all of them, or just the ones that we want to display or handle in some way. Obviously it depends on the amount. If we are obtaining one hundred of them, it is something that we can manage directly in memory, but if we are handling millions, we have to deal with them in a different way.

This is the case when a mechanism such as lazy loading makes things easier.

Vaadin components, like Grid or ComboBox, already support this kind of behavior, without the need of doing something special, but it only will take care of retrieving what is needed in a lazy loading fashion between the web browser and your web application.

Given that Vaadin only takes care of the presentation layer of your application, you need to do something to apply this behavior from your underlying data source (database, REST API, etc.).

According to the official documentation, you need to ask to your backend two questions:

Can you tell me the total amount of items that we are dealing with?

Of those, can you only give me the first N items, skipping the previous M items?

You don’t need to calculate N and M, those numbers are provided by the framework. They are basically the limit (N) and offset (M). You answer those two questions by providing two lambda expressions to the method DataProvider.fromCallbacks(), something like this:

DataProvider<Person, Void> dataProvider = DataProvider.fromCallbacks(

        // First callback fetches items based on a query

        query -> {

            // The index of the first item to load

            int offset = query.getOffset();

            // The number of items to load

            int limit = query.getLimit();

            List<Person> persons = getPersonService()

                    .fetchPersons(offset, limit);

            return persons.stream();

        },

        // Second callback fetches the number of items for a query

        query -> getPersonService().getPersonCount());

);

Grid<Person> grid = new Grid<>();

grid.setDataProvider(dataProvider);

It’s a rather common request to narrow down the size of the queried items, and one typical way of doing that is to make grids filterable.

But before digging into this, we need to define what is a Filter. For us it’s just a class that will contain values that we want to be taken into account when deciding if a given item will be returned or not.

In our case, let’s assume that we only want to filter people by their name and last name:

public class PersonFilter {

 

 private String nameFilter = null;

 private String lastNameFilter = null;



 public PersonFilter() { 

 }

 

 public PersonFilter(String nameFilter, String lastNameFilter) {

  this.setNameFilter(nameFilter);

  this.setLastNameFilter(lastNameFilter);

 }



 public String getNameFilter() {

  return nameFilter;

 }



 public String getLastNameFilter() {

  return lastNameFilter;

 }



 public void setNameFilter(String nameFilter) {

  this.nameFilter = nameFilter;

 }



 public void setLastNameFilter(String lastNameFilter) {

  this.lastNameFilter = lastNameFilter;

 }

}

Now that we have our filter, how to do the actual filtering?

        DataProvider<Person, PersonFilter> dataProvider =

            DataProvider.fromFilteringCallbacks(

            query -> {

              Optional<PersonFilter> filter = query.getFilter();

              return getPersonService().fetchPersons(

                query.getOffset(),

                query.getLimit(),

                filter.map(f -> f.getNameFilter()).orElse(null),

                filter.map(f -> f.getLastNameFilter()).orElse(null)

              );

            },

            query -> {

              Optional<PersonFilter> filter = query.getFilter();

              return getPersonService().getPersonCount(

                filter.map(f -> f.getNameFilter()).orElse(null),

                filter.map(f -> f.getLastNameFilter()).orElse(null)

              );

            }

          );

In this case, what we are doing, is just to obtain the filter from the query, and send the filter values to the backend class. We cannot do the filtering by ourselves because if the underlying backend is a database, then those filters have to be taken into account when building the SQL queries. If we do the filtering by ourselves, then it’s impossible to calculate the final count of the items, without retrieving all of the entities from the backend, causing our lazy loading mechanism to fail its goal (not to retrieve everything). The same applies when calling a remote API.

But let’s continue, if we want to change the filters dynamically, then we need to be able to configure the filters, for that we need to call the method withConfigurableFilter(), that will return an enhanced DataProvider with a method that will allow us to set a given Filter instance:

        PersonFilter gridFilter = new PersonFilter();

        ConfigurableFilterDataProvider<Person,Void,PersonFilter> dp = dataProvider.withConfigurableFilter();

        dp.setFilter(gridFilter);

We can change the filter later, or just save a reference to this instance and modify it’s values. After doing that a call to the method refreshAll() in the DataProvider will trigger the loading of the data with the new filter values.

Now, what about the visual part? … let’s see:

        HeaderRow hr = personsGrid.prependHeaderRow();

        TextField nameFilterTF = new TextField();

        nameFilterTF.addValueChangeListener(ev->{

         gridFilter.setNameFilter(ev.getValue());

         dp.refreshAll();

        });

        hr.getCell(personsGrid.getColumnByKey("name")).setComponent(nameFilterTF);

        TextField lastNameFilterTF = new TextField();

        lastNameFilterTF.addValueChangeListener(ev->{

         gridFilter.setLastNameFilter(ev.getValue());

         dp.refreshAll();

        });

        hr.getCell(personsGrid.getColumnByKey("lastName")).setComponent(lastNameFilterTF);

First we are creating a HeaderRow, that is a row that will show at the top of our fancy grid, that will contain components that will filter our data.

Then we are creating a TextField for holding the strings to filter the name and last name of our people.

Finally we are adding some value change listeners that will actually apply the filters when changing the value.

That’s great! … now we are loading filtered data in a lazy way using multiple filters.

But what if we need to sort the data? … let’s continue.

public class PersonSort {



 private String propertyName;

 private boolean descending;

 

 public String getPropertyName() {

  return propertyName;

 }

 public void setPropertyName(String propertyName) {

  this.propertyName = propertyName;

 }

 public boolean isDescending() {

  return descending;

 }

 public void setDescending(boolean descending) {

  this.descending = descending;

 }

  

}

The class is pretty self-explained. Just holding the property that is currently being sorted, and a boolean for specifying if we sort up or down.

You might think why we are constructing these classes (PersonFilter and PersonSort), given that we could just pass the values directly to the service. The main reason is to have framework agnostic classes to hold these kind of information and then pass them to the backend, making a cleaner separation of layers.

In our example, we are going to provide a new parameter to our PersonService: a list of PersonSort (sort orders). This is a list, because the order of the sort criteria is important, you can establish this order in the Grid component, by calling personsGrid.setMultiSort(true). Let’s review the changes in our code:

        DataProvider<Person, PersonFilter> dataProvider =

          DataProvider.fromFilteringCallbacks(

          query -> {

            Optional<PersonFilter> filter = query.getFilter();

            List<PersonSort> sortOrders = query.getSortOrders().stream().map(sortOrder->new PersonSort(sortOrder.getSorted(),sortOrder.getDirection().equals(SortDirection.ASCENDING))).collect(Collectors.toList());

            return getPersonService().fetchPersons(

              query.getOffset(),

              query.getLimit(),

              filter.map(f -> f.getNameFilter()).orElse(null),

              filter.map(f -> f.getLastNameFilter()).orElse(null),

              sortOrders

            );

          },

          query -> {

            Optional<PersonFilter> filter = query.getFilter();

            return getPersonService().getPersonCount(

              filter.map(f -> f.getNameFilter()).orElse(null),

              filter.map(f -> f.getLastNameFilter()).orElse(null)

            );

          }

        );

Now we are assembling this list of PersonSort, using query.getSortOrders().

The implementation of PersonService is irrelevant, because it’s up to the backend to provide the data requested using the information provided:

If the backend obtains the data from a relational database, then a SQL clause has to be created dynamically based on that information. For example:

SELECT *

FROM PERSONS P

WHERE P.NAME LIKE %FILTERNAME% AND P.LASTNAME LIKE %LASTNAME%

ORDER BY P.NAME DESC, P.LASTNAME ASC

LIMIT OFFSET,LIMIT