Wednesday, June 03, 2015

JSON Data - The Superglue that Holds the Mobile Universe Together

Peter Rogers
Digital Consultant, Cognizant
My friend and a very technical mobility expert, Peter Rogers, shares his insights into mobile application development and connecting mobile applications to backend data sources in this guest blog.  Enjoy!
*****
These days XML is just a good or bad memory and the world has moved to JSON data. The one part that got lost in translation however was XML Schemas. This leaves us without a way of validating a lump of data loaded across the wire other than using custom code based around a published API…or does it...Many developers and architects will tell you they don’t miss Schemas and even deride the whole concept, mostly just pointing you at an API, and then waving their hands is surprise when the client application falls over after an unexpected data change on the server. I have seen enough digital projects hit rocky waters to realise that a strategy is required to handle JSON effectively.

There are two strategies which are suitable for different environments:
1.       Find a JSON Schema standard and define a tight contract between client and server
2.       Find a JSON standard for defining a loose contract between client and server

JSON Schemas is a (sort of) standard that has existed for quite a while and is currently up to draft 4. These offer a similar capability to XML Schemas in that you can validate the structure of a JSON data lump including specifying the type of data and validation routines. The most useful of these validation rules are required fields and maximum/minimum number of items in an array. JSON Schemas are JSON documents themselves and so it is best to view them in a JSON visualisation tool of which there are many available online (I personally use http://jsonviewer.stack.hu/).

The JSON Schema V4 draft can be found here (http://tools.ietf.org/html/draft-zyp-json-schema-04) and there are quite a few validators online too ((http://jsonschemalint.com/draft4/ and http://www.jsonschemavalidator.net/). The best guide I have found on the subject is probably (http://spacetelescope.github.io/understanding-json-schema/UnderstandingJSONSchema.pdf) and it will be quite possible to start writing your own JSON Schemas after a few hours.

{
            "$schema": "http://json-schema.org/draft-04/schema#",
            "type": "object",
            "properties": {
                        "name": {"type": "string"},
                        "address":{"type": "string"}
            },
            required: ["name", "address"]
}

You can find libraries for iOS and Android, and if you want to compare library performance then there is a good benchmark here (https://github.com/ebdrup/json-schema-benchmark). The homepage can be found here (http://json-schema.org/) and there is an interesting alternative approach called Indicative that uses a flat form of schema validation (http://amanvirk.me/validating-models-in-node-js-using-indicative/). The one thing to watch is that your chosen library is using Draft 4 and not Draft 3. The best way to use JSON Schemas in my view is to define different levels of strictness for your JSON coming over the wire. That way you can have a strict mode which rejects anything that is not the complete data structure and more tolerate modes which allow an agreed minimum level of data. This allows data recovery solutions for when most of the data is OK but a small part is missing. This is achieved through multiple Schemas with different definitions of the required fields and different levels of field validations.

You will often find a lot of push back when you mention JSON Schemas, often from those who are on the server side because they can feel the noose tightening around their neck. Luckily there is another option of using a loose data contract in which you merely specify which data fields are mandatory. This is far more suitable to situations where you have less authority over the server side, you are looking at maximal data recoverability or where the minimum JSON Schema would not be worth the effort. The question is how we can specify the mandatory data fields in relation to the data structure. The answer is JsonPath which is a JSON flavour of the old XPath standard.

JsonPath expressions always refer to a JSON structure in the same way as XPath expressions are used in combination with an XML document. The root object in JsonPath is referred to as $.  JsonPath expressions can use the dot notation or the bracket notation but the latter is far more ugly to be frank. There are a number of operators ($, @, *,  ..) which can provide much more flexibility to the syntax and there is also the ability to create inline predicates, filter predicates and custom predicates. If a path is indefinite then it will return a list of items as opposed to a single result.

$.petstore.animal[0].name

$..book[?(@.price < 10)]

Even though it was specified by Stefan Goessner back in 2007, it has found new love recently in a Java port called Jayway JsonPath and a JavaScript/Node library which are found at (https://github.com/s3u/JSONPath and https://github.com/jayway/JsonPath). JsonPath is a nice solution because it allows us to specify a number of mandatory fields with relation to their containing data structure.  We can store these inside a configuration file and validate they return a data item either using a JSON parser and code routines on iOS, or with an actual Java library directly on Android.
One interesting approach is to have your UX team define a minimum viable user interface for each screen, which is then translated into mandatory data fields using JsonPath and then validated in client side code.

Hopefully this article has given you more tools to handle JSON data contracts between client and server without relying too heavily on mental telepathy between developers.

************************************************************************
Kevin Benedict
Writer, Speaker, Senior Analyst
Digital Transformation, EBA, Center for the Future of Work Cognizant
View my profile on LinkedIn
Read more at Future of Work
Learn about mobile strategies at MobileEnterpriseStrategies.com
Follow me on Twitter @krbenedict
Subscribe to Kevin'sYouTube Channel
Join the Linkedin Group Strategic Enterprise Mobility
Join the Google+ Community Mobile Enterprise Strategies

***Full Disclosure: These are my personal opinions. No company is silly enough to claim them. I am a mobility and digital transformation analyst, consultant and writer. I work with and have worked with many of the companies mentioned in my articles.