{"id":286,"date":"2019-10-31T19:59:36","date_gmt":"2019-10-31T18:59:36","guid":{"rendered":"http:\/\/xpam.pl\/blog\/?p=286"},"modified":"2024-04-02T23:35:52","modified_gmt":"2024-04-02T21:35:52","slug":"receive-only-the-data-your-client-needs-full-dynamic-json-filtering-with-jackson","status":"publish","type":"post","link":"https:\/\/xpam.pl\/blog\/?p=286","title":{"rendered":"Receive only the data your client needs – full dynamic JSON filtering with Jackson"},"content":{"rendered":"

A lot of times JSON returned by your REST API grows to incredibly big structures and data sizes due to business logic complexity that is added over time. Then there are API methods returning a list of objects which can be huge in size. If you serve multiple clients, each one can have different demands on what is and is not needed from that data so backend can't decide on it's own what to prune and what to keep. Ideally, backend would always return full JSON by default but allow clients to specify exactly what they want and have backend adjust the response accordingly. We can achieve this using the power of Jackson<\/a> library.<\/p>\n

Goal<\/strong>:
\n– allow REST API clients to decide on their own which parts of JSON to receive (full JSON filtering)<\/p>\n

Resources for this tutorial<\/strong>:
\n– Microprofile or JakartaEE platform (JAX-RS)
\n– Jackson library
\n– Java classes (lib) representing your API responses which are serialized to JSON
\n– some custom code to bring things together<\/p>\n

The lib module<\/h2>\n

First lets define a few classes which represent our JSON responses.<\/p>\n

public class Car {\n\n  private Engine engine;\n\n  private List<Wheel> wheels;\n\n  private String brand;\n\n \/\/Getters and setters..\n}\n\npublic class Wheel {\n\n  private BigDecimal pressure;\n\n  \/\/Getters and setters..\n}\n\npublic class Engine {\n  \n  private int numOfCylinders;\n\n  private int hp;\n\n  \/\/Getters and setters..\n}<\/code><\/pre>\n
Our lib serialized to JSON would look something like this:<\/p>\n
{\n    "engine": {\n        "numOfCylinders": 4,\n        "hp": 180\n    },\n    "wheels": [\n        {\n            "pressure": 30.2\n        },\n        {\n            "pressure": 30.1\n        },\n        {\n            "pressure": 30.0\n        },\n        {\n            "pressure": 30.3\n        }\n    ],\n    "brand": "Jugular"\n}<\/code><\/pre>\n
Let's say one of our clients only needs the engine's horse power and brand information. We want to be able to specify a query parameter like filter=car:engine,brand;engine:hp <\/code>and receive the following:<\/p>\n
{\n    "engine": {\n        "hp": 180\n    },\n    "brand": "Jugular"\n}<\/code><\/pre>\n
Step in Jackson<\/h2>\n
Jackson provides an annotation for such tasks called @JsonFilter<\/code>. This annotation expects a filter name as a parameter and a named filter must be applied to serialization mapper, for example:<\/p>\n
FilterProvider filters = new SimpleFilterProvider()\n.addFilter("carFilter", SimpleBeanPropertyFilter.filterOutAllExcept("wheels"));      \nString jsonString = mapper.writer(filters)...<\/code><\/pre>\n
As you can see, all we need is already there but is a rather static affair. We need to take this and make it fully dynamic and client driven.<\/p>\n
The reason filter needs a name is because each one is bound to a class and attribute filtering is done on that class. What we need to do is transform car:engine,brand<\/code> into a carFilter<\/code> and SimpleBeanPropertyFilter.filterOutAllExcept("engine", "brand")<\/code>.<\/p>\n
For starters, lets add the filters to our classes:<\/p>\n
@JsonFilter("carFilter")\npublic class Car {}\n\n@JsonFilter("engineFilter")\npublic class Engine {}\n\n@JsonFilter("wheelFilter")\npublic class Wheel {}<\/code><\/pre>\n
There is one thing about this that bothers me.. the filter name is a static String so it is refactor unfriendly if class name changes some day. Couldn't we just name the filters by taking a look at the name of the underlying class? Yes we can, by extending Jackson introspection:<\/p>\n
public class MyJacksonAnnotationIntrospector extends JacksonAnnotationIntrospector {\n\n    @Override\n    public Object findFilterId(Annotated a) {\n        JsonFilter ann = _findAnnotation(a, JsonFilter.class);\n        if (ann != null) {\n            String id = ann.value();\n            if (id.length() > 0) {\n                return id;\n            }\n            else {\n                try {\n                    \/\/Use className+Filter as filter ID if ID is not set, e.g. Car -> carFilter\n                    Class<?> clazz = Class.forName(a.getName());\n                    return StringUtils.uncapitalize(clazz.getSimpleName())+"Filter";\n                } catch (ClassNotFoundException e) {\n                    e.printStackTrace();\n                }\n            }\n        }\n        return null;\n    }\n}<\/code><\/pre>\n
With this, any class annotated with @JsonFilter(\"\") will automatically get a filter called classNameFilter<\/strong><\/em>. We no longer need to specify filter names and keep them in sync with class names.<\/p>\n
Our lib now looks like:<\/p>\n
@JsonFilter("")\npublic class Car {}\n\n@JsonFilter("")\npublic class Engine {}\n\n@JsonFilter("")\npublic class Wheel {}<\/code><\/pre>\n
Next step is to transform and apply the query parameters into our filter structure.<\/p>\n
First, register a Jackson provider for JAX-RS server:<\/p>\n
@Provider\npublic class JacksonProvider extends JacksonJsonProvider implements ContextResolver<ObjectMapper> {\n    \n    private final ObjectMapper mapper;\n\n    public JacksonProvider() {\n        mapper = new ObjectMapper();\n        mapper.registerModule(new JavaTimeModule());\n        mapper.setFilterProvider(new SimpleFilterProvider().setFailOnUnknownId(false));\n        mapper.setAnnotationIntrospector(new MyJacksonAnnotationIntrospector());\n    }\n\n    @Override\n    public ObjectMapper getContext(Class<?> type) {\n        return mapper;\n    }\n}<\/code><\/pre>\n
We register our own introspector and disable failures on unknown filters (in case client filters by something nonexisting).<\/p>\n
Provider must be registered in your rest Application.<\/p>\n
@ApplicationPath("")\npublic class MyApplication extends Application {\n\n    @Override\n    public Set<Class<?>> getClasses() {\n\n        Set<Class<?>> classes = new HashSet<>();\n\n        classes.add(JacksonProvider.class);\n\n        return classes;\n    }\n}<\/code><\/pre>\n
Finally, we implement our own MessageBodyWriter to override the default serialization and apply the filters dynamically.<\/p>\n
@Provider\n@Produces(MediaType.APPLICATION_JSON)\npublic class JsonFilterProvider implements MessageBodyWriter<Object>; {\n\n    @Context\n    private UriInfo uriInfo;\n\n    @Context\n    private JacksonProvider jsonProvider;\n\n    public static final String PARAM_NAME = "filter";\n\n    public boolean isWriteable(Class<?> aClass, Type type, Annotation[] annotations, MediaType mediaType) {\n        return MediaType.APPLICATION_JSON_TYPE.equals(mediaType);\n    }\n\n    public long getSize(Object object, Class<?> aClass, Type type, Annotation[] annotations,\n                        MediaType mediaType) {\n        return -1;\n    }\n\n    public void writeTo(Object object, Class<?> aClass, Type type, Annotation[] annotations,\n                        MediaType mediaType, MultivaluedMap<String, Object> stringObjectMultivaluedMap,\n                        OutputStream outputStream) throws IOException, WebApplicationException {\n\n        String queryParamValue = uriInfo.getQueryParameters().getFirst(PARAM_NAME);\n        if (queryParamValue!=null && !queryParamValue.equals("")) {\n\n            SimpleFilterProvider sfp = new SimpleFilterProvider().setFailOnUnknownId(false);\n\n            \/\/We link @JsonFilter annotation with dynamic property filter\n            for (Map.Entry<String, Set<String>> entry : getFilterLogic(queryParamValue).entrySet()) {\n                sfp.addFilter(entry.getKey() + "Filter", SimpleBeanPropertyFilter.filterOutAllExcept(entry.getValue()));\n            }\n\n            jsonProvider.locateMapper(aClass, mediaType).writer(sfp).writeValue(outputStream, object);\n        }\n        else {\n            jsonProvider.locateMapper(aClass, mediaType).writeValue(outputStream, object);\n        }\n    }\n\n    \/\/Map of object names and set of fields\n    private Map<String, Set<String>> getFilterLogic(String paramValue) {\n        \/\/ ?jsonFilter=car:engine,brand;engine:numOfCylinders\n        String[] filters = paramValue.split(";");\n\n        Map<String, Set<String>> filterAndFields = new HashMap<>();\n\n        for (String filterInstance : filters) {\n            \/\/car:engine,brand\n            List<String> pair = Arrays.asList(filterInstance.split(":"));\n            if (pair.size()!=2) {\n                throw new RuntimeException();\n            }\n\n            Set<String> fields = new HashSet<>(Arrays.asList(pair.get(1).split(",")));\n            filterAndFields.put(pair.get(0), fields);\n        }\n\n        return filterAndFields;\n    }\n}<\/code><\/pre>\n
getFilterLogic<\/code> method assembles the query parameter structure into a map of <String className, Set<String> fields> which is then applied as a Jackson filter.<\/p>\n
Finally, we need to register our JsonFilterProvider<\/code> in our Application as we did with JacksonProvider.<\/p>\n
@ApplicationPath("")\npublic class MyApplication extends Application {\n\n    @Override\n    public Set<Class<?>> getClasses() {\n\n        Set<Class<?>> classes = new HashSet<>();\n\n        classes.add(JacksonProvider.class);\n        classes.add(JsonFilterProvider.class);\n\n        return classes;\n    }\n}<\/code><\/pre>\n
One small deficiency with this solution is that once you specify a class with fields to filter, it will be filtered wherever in the nested JSON structure it appears, you can't just filter a specific class at a specific level. Realistically, I think this is a rather minor problem compared to the benefits and the simplicity of the implementation.<\/p>\n
Finally a question on documentation. How do you tell the client developer about all the possible filter object names and their attributes? If you use OpenAPI you are 95% there. Simply document that you can filter by model name followed by attribute name. Client developer can easily figure out the names from your OpenAPI specification. The only remaining problem is when you don't want to allow filtering on all classes. In this case my approach would be to document a filterable class in OpenAPI description:<\/p>\n
@ApiModel(description = "[Filterable] A car.")<\/code><\/pre>\n
This manual approach of documenting goes against the rest of the paradigm so a real purist would write an OpenAPI extension that would introspect all @JsonFilter annotations and modify the descriptions automatically. But let's leave that for a future blog post.<\/p>\n
 <\/p>\n
A similar, more advanced and out-of-the-box solution is squiggly<\/a>, which also uses Jackson under the hood.<\/p>\n
 <\/p>\n
\r\n
\r\n
\r\n Cen
\r\nGitHub<\/a>
\r\nEurobattle.net<\/a>
\r\nLagabuse.com<\/a>
\r\nBnetdocs<\/a>
\r\n<\/div>\r\n","protected":false},"excerpt":{"rendered":"
A lot of times JSON returned by your REST API grows to incredibly big structures and data sizes due to business logic complexity that is added over time. Then there are API methods returning a list of objects which can be huge in size. If you serve multiple clients, each one can have different demands […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30,3],"tags":[],"class_list":["post-286","post","type-post","status-publish","format-standard","hentry","category-java","category-programming"],"_links":{"self":[{"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/286","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=286"}],"version-history":[{"count":20,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/286\/revisions"}],"predecessor-version":[{"id":305,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/286\/revisions\/305"}],"wp:attachment":[{"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/xpam.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}