JSON Persistence
A major part of TMS FNC Core is the ability to save and load objects to and from JSON. There is a complete JSON parser, reader and writer included. Components inheriting from the base TTMSFNCCustomControl, TTMSFNCCustomComponent class and including the unit (FMX.)(VCL.)(WEBLib.)(LCL)TMSFNCPersistence.pas get JSON persistence by default. Even when only using TMS FNC Core, objects that are not directly related to FNC can be persisted with a couple of lines of code.
Reading through these topics will focus on a TPerson class and eventually lead into a class structure that has all the bits and pieces together required for JSON persistence. Note that when referring to unit names, the prefix of the framework ((FMX.)(VCL.)(WEBLib.)(LCL)) will not be included, for readability purposes. The code snippets are written in FMX as a default framework, but can easily be ported to other frameworks.
Basics
Units
After installing TMS FNC Core, the most important units are the TMSFNCPersistence.pas and the TMSFNCTypes.pas unit. There are a couple of class helper functions in the TMSFNCUtils unit, but that is of lesser importance and mostly revolve around parsing raw JSON content.
TMSFNCPersistence.pas
: Unit containing interfaces and class functions to save and load JSON data to and from objectsTMSFNCTypes.pas
: Unit containing some class helpers to assign JSON data, which internally map on the TMSFNCPersistence.pas unit.
TMSFNCPersistence.pas is a unit containing 2 important classes, TTMSFNCObjectPersistence and TTMSFNCPersistence. TTMSFNCObjectPersistence has a class function to save and a class procedure to load JSON data to and from an object.
TTMSFNCObjectPersistence = class
public
class function SaveObjectToString(AObject: TObject): string;
class procedure LoadObjectFromString(AObject: TObject; AString: string);
end;
TTMSFNCPersistence is the base class that provides a lot of functionality to detect property names, types and the ability to read and write custom properties.
Published properties
Before we get started, there is one important requirement to make sure that our object and its properties get persisted. Only published properties will be persisted. Internally, a cross-platform method is used and currently maps on published properties only due to technical reasons. This might expand to public properties as well in the future, but for now, it's important to move properties to the published section in order to persist them, the same way as is required when creating components that have properties persisted in the form file.
Basic object persistence: TPerson
Let's get started by defining our TPerson class
type
TPersonAddress = class(TPersistent)
private
FPostalCode: string;
FAddressLocality: string;
FAddressRegion: string;
FStreetAddress: string;
published
property AddressLocality: string read FAddressLocality write FAddressLocality;
property AddressRegion: string read FAddressRegion write FAddressRegion;
property PostalCode: string read FPostalCode write FPostalCode;
property StreetAddress: string read FStreetAddress write FStreetAddress;
end;
TPerson = class(TPersistent)
private
FAddress: TPersonAddress;
FColleagues: TStringList;
FBirthDate: string;
FName: string;
FEmail: string;
FTelephone: string;
FGender: string;
FNationality: string;
FJobTitle: string;
FURL: string;
public
constructor Create;
destructor Destroy; override;
published
property Address: TPersonAddress read FAddress;
property Colleagues: TStringList read FColleagues;
property Email: string read FEmail write FEmail;
property JobTitle: string read FJobTitle write FJobTitle;
property Name: string read FName write FName;
property BirthDate: string read FBirthDate write FBirthDate;
property Gender: string read FGender write FGender;
property Nationality: string read FNationality write FNationality;
property Telephone: string read FTelephone write FTelephone;
property URL: string read FURL write FURL;
end;
Inspecting TPerson
When creating an instance of TPerson, there is a nice utility helper method that can log an object in JSON to the IDE debug output window. When including the TMSFNCTypes unit, call [Object].Log; to output the JSON as shown in the following sample.
The output of the object in JSON will be rendered in one line. To view it properly, there are a lot of online JSON "beautifiers" available that take care of parsing and rendering in a readable way. Rendering our object after initializing it, will show the JSON structure and also, that there is no info available as there are no properties set. Changing properties on our created TPerson object, will of course have impact on the output of our object.
{
"$type": "TPerson",
"Address": {
"$type": "TPersonAddress",
"AddressLocality": "",
"AddressRegion": "",
"PostalCode": "",
"StreetAddress": ""
},
"BirthDate": "",
"Colleagues": [],
"Email": "",
"Gender": "",
"JobTitle": "",
"Name": "",
"Nationality": "",
"Telephone": "",
"URL": ""
}
Loading Data
For testing purposes, we have predefined a jsonSample constant which already contains JSON data ready to be loaded into the TPerson object.
const
jsonSample =
'{' +
'"$type": "TPerson",' +
'"address":{' +
'"$type": "TPersonAddress",' +
'"addressLocality":"Colorado Springs",' +
'"addressRegion":"CO",' +
'"postalCode":"80840",' +
'"streetAddress":"100 Main Street"' +
'},' +
'"colleagues":[' +
'"http://www.example.com/JohnColleague.html",' +
'"http://www.example.com/JameColleague.html"' +
'],' +
'"email":"info@example.com",' +
'"jobTitle":"Research Assistant",' +
'"name":"Jane Doe",' +
'"birthDate":"1979-10-12",' +
'"gender":"female",' +
'"nationality":"Albanian",' +
'"telephone":"(123) 456-6789",' +
'"url":"http://www.example.com"' +
'}';
As explained there are multiple ways to map the JSON data onto the TPerson object.
1. Use the TTMSFNCPersistence class
Add the unit TMSFNCPersistence to the uses list ( = FMX., LCL, VCL., WEBLib.), and use the following code:
var
p: TPerson;
s: TStringStream;
begin
p := TPerson.Create;
s := TStringStream.Create(jsonSample);
try
TTMSFNCPersistence.LoadSettingsFromStream(p, s);
finally
s.Free;
p.Free;
end;
end;
2. Use the TTMSFNCObjectPersistence class
An alternative is to use the class TTMSFNCObjectPersistence that maps JSON to the object. (string variable or constant only)
var
p: TPerson;
begin
p := TPerson.Create;
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
finally
p.Free;
end;
end;
3. Use the object class helper in *TMSFNCTypes unit
An alternative is to use the class helper that maps JSON to the object via a class helper. (string variable or constant only)
Saving Data
Writing to JSON is as easy as reading. Simply use SaveSettingsToFile or SaveSettingsToStream or use the JSON object class helper to get the JSON from the object. For each of the above three load methods, there are an equivalent in save methods as well. For testing purposes, we want to change the name of "Jane Doe", to "Joe Heart" first to demonstrates how the result of the JSON output changes accordingly.
var
p: TPerson;
begin
p := TPerson.Create;
try
p.JSON := jsonSample;
p.Name := 'Joe Heart';
TTMSFNCUtils.Log(p.JSON);
//or
TTMSFNCPersistence.SaveSettingsToFile(p, 'TPerson.json');
finally
p.Free;
end;
end;
Alternatively, the TTMSFNCObjectPersistence class can be used to save the object to a JSON string
var
p: TPerson;
j: string;
begin
p := TPerson.Create;
try
j := TTMSFNCObjectPersistence.SaveObjectToString(p);
finally
p.Free;
end;
end;
Or by using the TMSFNCTypes unit class helpers
$type property
The above data loading expects the TPerson object to exist and be accessible. In some situations however, the object will not be available when loading the JSON data, or some situations might expect the object to be loaded alongside the JSON data. As you might already have noticed, in the JSON output of the object, there are properties named "$type", which map on the classes of the objects in JSON. These properties are used for the JSON persistence in FNC to be able to instantiate those classes. To make sure the object(s) can be created when required, it's good practice to register your classes. In case of our TPerson class, this translates to:
Especially in case of collections and generics, the classname of the objects are used, but this is covered in the next part of this blog series, where we are going to handle collections.
Collections
In the previous chapter, we talked about the basics for saving and loading objects to and from JSON data. Today, we are going a step further and will look at collections. To demonstrate this, we are going to add a TPersonRelations collection and a TPersonRelation TCollectionItem to our TPerson class.
TPersonRelation = class(TCollectionItem)
private
FName: string;
FDescription: string;
published
property Name: string read FName write FName;
property Description: string read FDescription write FDescription;
end;
TPersonRelations = class(TCollection)
private
function GetItem(Index: Integer): TPersonRelation;
procedure SetItem(Index: Integer; const Value: TPersonRelation);
public
constructor Create;
property Items[Index: Integer]: TPersonRelation read GetItem write SetItem; default;
end;
TPerson = class(TPersistent)
private
FAddress: TPersonAddress;
FColleagues: TStringList;
FBirthDate: string;
FName: string;
FEmail: string;
FTelephone: string;
FGender: string;
FNationality: string;
FJobTitle: string;
FURL: string;
FRelations: TPersonRelations;
public
constructor Create;
destructor Destroy; override;
published
property Address: TPersonAddress read FAddress;
property Colleagues: TStringList read FColleagues;
property Email: string read FEmail write FEmail;
property JobTitle: string read FJobTitle write FJobTitle;
property Name: string read FName write FName;
property BirthDate: string read FBirthDate write FBirthDate;
property Gender: string read FGender write FGender;
property Nationality: string read FNationality write FNationality;
property Telephone: string read FTelephone write FTelephone;
property URL: string read FURL write FURL;
property Relations: TPersonRelations read FRelations;
end;
var
p: TPerson;
r: TPersonRelation;
begin
p := TPerson.Create;
try
p.JSON := jsonSample;
r := p.Relations.Add;
r.Name := 'John Doe';
r.Description := 'Brother';
r := p.Relations.Add;
r.Name := 'Mia Reyes';
r.Description := 'Mother';
p.Log;
finally
p.Free;
end;
end;
Saving a collection
The output of the log statement is
{
"$type": "TPerson",
"Address": {
"$type": "TPersonAddress",
"AddressLocality": "Colorado Springs",
"AddressRegion": "CO",
"PostalCode": "80840",
"StreetAddress": "100 Main Street"
},
"BirthDate": "1979-10-12",
"Colleagues": [],
"Email": "info@example.com",
"Gender": "female",
"JobTitle": "Research Assistant",
"Name": "Jane Doe",
"Nationality": "Albanian",
"Relations": [
{
"$type": "TPersonRelation",
"Description": "Brother",
"Name": "John Doe"
},
{
"$type": "TPersonRelation",
"Description": "Mother",
"Name": "Mia Reyes"
}
],
"Telephone": "(123) 456-6789",
"URL": "http://www.example.com"
}
As you can see, the Relations property of type TPersonRelations is generated as an array of JSON objects, each object represents a TCollectionItem of type TPersonRelation.
Loading a collection without "$type"
The object that loads the JSON defines the property type, which means that even when a JSON array can be loaded in a TStringList, a TList, or a TCollection, TPersonRelations is of type TCollection and the JSON array loading will be mapped on a TCollection. When a JSON object is loaded from inside the JSON array, the "$type" property defines the object type. In the basics chapter, we explain what the "$type" property does and why it is important to register your class. When changing our jsonSample const to include relations, but leaving out all "$type" properties the initial output of our TPerson object will have an empty relations collection.
const
jsonSample =
'{' +
'"address":{' +
'"addressLocality":"Colorado Springs",' +
'"addressRegion":"CO",' +
'"postalCode":"80840",' +
'"streetAddress":"100 Main Street"' +
'},' +
'"colleague":[' +
'"http://www.example.com/JohnColleague.html",' +
'"http://www.example.com/JameColleague.html"' +
'],' +
'"email":"info@example.com",' +
'"jobTitle":"Research Assistant",' +
'"name":"Jane Doe",' +
'"birthDate":"1979-10-12",' +
'"gender":"female",' +
'"nationality":"Albanian",' +
'"relations": ['+
'{'+
'"Description": "Brother",'+
'"Name": "John Doe"'+
'},'+
'{'+
'"Description": "Mother",'+
'"Name": "Mia Reyes"'+
'}'+
'],'+
'"telephone":"(123) 456-6789",' +
'"url":"http://www.example.com"' +
'}';
This is because the way the JSON is loaded. When using the class helpers, the "$type" property is ignored. Basically, it is adapted to make sure it can load any kind of JSON, whether it's JSON coming directly from a predefined object structure, or an unknown structure that needs to be mapped on the object, without knowing the class types of the JSON objects inside the JSON structure. To fix this, we need to implement an interface on our TPersonRelations collection class, named ITMSFNCBaseListIO.
ITMSFNCBaseListIO = interface
['{FAB1D21E-D798-4CE0-B17B-9D75E4456AB4}']
function GetItemClass: TClass;
end;
The ITMSFNCBaseListIO interface requests from the TCollection class, what the base class is for an item. When the "$type" property is missing, the interface can be used to return the correct class. Implementing this on our TPersonRelations collection implies the default interface implementation requirements as shown below.
TPersonRelations = class(TCollection, ITMSFNCBaseListIO)
private
function GetItem(Index: Integer): TPersonRelation;
procedure SetItem(Index: Integer; const Value: TPersonRelation);
function GetItemClass: TClass;
function QueryInterface(const IID: TGUID; out obj): HResult; stdcall;
function _AddRef: Integer; stdcall;
function _Release: Integer; stdcall;
public
constructor Create;
property Items[Index: Integer]: TPersonRelation read GetItem write SetItem; default;
function Add: TPersonRelation;
end;
Now, to make sure the item is created and is properly added to the collection, we need to add the ITMSFNCBasePersistenceIO interface.
ITMSFNCBasePersistenceIO = interface
['{91DEAFC3-8932-45F4-A3ED-5AAA0C0E9250}']
function CreateObject(const AClassName: string; const ABaseClass: TClass): TObject;
end;
This interface needs to be added to the root object, because the root object TPerson is our reference for any JSON saving and loading actions.
TPerson = class(TInterfacedPersistent, ITMSFNCBasePersistenceIO)
...
protected
function CreateObject(const AClassName: string; const ABaseClass: TClass): TObject;
public
...
function TPerson.CreateObject(const AClassName: string;
const ABaseClass: TClass): TObject;
begin
Result := nil;
if AClassName = 'TPersonRelation' then
Result := TPersonRelation.Create(Relations);
end;
To load the data, we can now use our class helper and make sure we register our TPersonRelation class.
var
p: TPerson;
begin
p := TPerson.Create;
try
p.JSON := jsonSample;
p.Log;
finally
p.Free;
end;
end;
Loading a collection with "$type"
When our JSON sample data contains the "$type" properties for each object, including the root object, it's not required to define the ITMSFNCBaseListIO and ITMSFNCBasePersistenceIO interfaces to load the data. The data can be mapped directly on the object, but the class helpers cannot be used as they will ignore the "$type" properties. The code will change to
var
p: TPerson;
begin
p := TPerson.Create;
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
p.Log;
finally
p.Free;
end;
end;
Generics
The previous chapter explained how to deal with collections. This chapter will use and transform the TPersonRelations collection to a TObjectList
TPersonRelation = class(TPersistent)
private
FName: string;
FDescription: string;
public
procedure Assign(Source: TPersistent); override;
published
property Name: string read FName write FName;
property Description: string read FDescription write FDescription;
end;
TPersonRelations = TObjectList<TPersonRelation>;
const
jsonSample =
'{' +
'"$type":"TPerson",' +
'"address":{' +
'"$type":"TPersonAddress",' +
'"addressLocality":"Colorado Springs",' +
'"addressRegion":"CO",' +
'"postalCode":"80840",' +
'"streetAddress":"100 Main Street"' +
'},' +
'"colleague":[' +
'"http://www.example.com/JohnColleague.html",' +
'"http://www.example.com/JameColleague.html"' +
'],' +
'"email":"info@example.com",' +
'"jobTitle":"Research Assistant",' +
'"name":"Jane Doe",' +
'"birthDate":"1979-10-12",' +
'"gender":"female",' +
'"nationality":"Albanian",' +
'"relations": ['+
'{'+
'"$type":"TPersonRelation",' +
'"Description": "Brother",'+
'"Name": "John Doe"'+
'},'+
'{'+
'"$type":"TPersonRelation",' +
'"Description": "Mother",'+
'"Name": "Mia Reyes"'+
'}'+
'],'+
'"telephone":"(123) 456-6789",' +
'"url":"http://www.example.com"' +
'}';
Saving a generic list
Saving an object including a generic list is as simple as using the class helpers, in the same way as you would do in the previous chapter related to collections. The generic list (or object list), will be transformed in a JSON array of objects, including the class type of the TPersonRelation object. Instead of a TCollectionItem, this is now a TPersistent class.
The output of this statement is{
"$type": "TPerson",
"Address": {
"$type": "TPersonAddress",
"AddressLocality": "Colorado Springs",
"AddressRegion": "CO",
"PostalCode": "80840",
"StreetAddress": "100 Main Street"
},
"BirthDate": "1979-10-12",
"Colleagues": [],
"Email": "info@example.com",
"Gender": "female",
"JobTitle": "Research Assistant",
"Name": "Jane Doe",
"Nationality": "Albanian",
"Relations": [
{
"$type": "TPersonRelation",
"Description": "Brother",
"Name": "John Doe"
},
{
"$type": "TPersonRelation",
"Description": "Mother",
"Name": "Mia Reyes"
}
],
"Telephone": "(123) 456-6789",
"URL": "http://www.example.com"
}
Loading a generic list
To understand how to load JSON data containing an array which will be mapped onto a generic list, please head over to the previous chapter explaining which interfaces are required to handle and load JSON with and without the "$type" properties. The loading mechanism for a generic list is exactly the same as with a TCollection.
Working with a TDictionary
TDictionary
TMyObject = class(TPersistent)
private
FMyProperty: string;
published
property MyProperty: string read FMyProperty write FMyProperty;
end;
TMyDictionary = TObjectDictionary<string, TMyObject>;
var
d: TMyDictionary;
o: TMyObject;
begin
d := TMyDictionary.Create;
try
o := TMyObject.Create;
o.MyProperty := 'Property Value 1';
d.Add('1', o);
o := TMyObject.Create;
o.MyProperty := 'Property Value 2';
d.Add('2', o);
d.Log;
finally
d.Free;
end;
[
{
"2": {
"$type": "TMyObject",
"MyProperty": "Property Value 2"
}
},
{
"1": {
"$type": "TMyObject",
"MyProperty": "Property Value 1"
}
}
]
Supported generic types
Not all types are supported when persisting to JSON. Below is a list of supported types that can be used when converting your object to JSON or vice versa.
TList<TObject>, or TObjectList<TObject>
TList<string>
TList<Integer>
TList<Double>
In the following sample, we'll demonstrates how to use and persist a generic TList
var
l: TMyIntegerList;
begin
try
l := TMyIntegerList.Create;
l.Add(50);
l.Add(6);
l.Add(978);
l.Add(20);
l.Log;
finally
l.Free;
end;
end;
Undo / Redo Manager
Getting started
To get started using this class, add the unit (FMX.)(VCL.)(WEBLib.)(LCL)TMSFNCUndo.pas to the uses list of your project. This enables you to make use of the TTMSFNCUndoManager class. The TTMSFNCUndoManager has a couple of public methods that can be used to navigate through the object history.
function NextUndoAction: string; //returns the next undo action
function NextRedoAction: string; //returns the next redo action
function CanUndo: Boolean; //returns a boolean if an undo action is possible
function CanRedo: Boolean; //returns a boolean if a redo action is possible
procedure Undo; //executes an undo action
procedure Redo; //executes a redo action
procedure ClearUndoStack; //clears the object history
procedure PushState(const {%H-}AActionName: string); //puts an object state on the history stack
property MaxStackCount: Integer read FMaxStackCount write FMaxStackCount default 20; //maximum number of history items on the stack
Each instance of the TTMSFNCUndoManager has a unique reference to the object it will manage, meaning that there can only be one manager per object. To instantiate the TTMSFNCUndoManager, call:
Initializing TPerson
Now let's get back to our TPerson implementation we did in the first chapter. We create a TPerson object, and load the JSON data with the known methods.
var
p: TPerson;
begin
p := TPerson.Create;
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
finally
p.Free;
end;
end;
Our TPerson object instance now contains the information from the predefined JSON data sample. We want to keep history of what happens with the TPerson object so we are going to create a TTMSFNCUndoManager instance, managing the object.
var
p: TPerson;
u: TTMSFNCUndoManager;
begin
p := TPerson.Create;
u := TTMSFNCUndoManager.Create(p);
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
p.Log;
finally
u.Free;
p.Free;
end;
end;
var
p: TPerson;
u: TTMSFNCUndoManager;
begin
p := TPerson.Create;
u := TTMSFNCUndoManager.Create(p);
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
u.PushState('init');
p.Name := 'ERROR';
u.PushState('error_data');
p.Log;
finally
u.Free;
p.Free;
end;
end;
{
"$type": "TPerson",
"Address": {
"$type": "TPersonAddress",
"AddressLocality": "Colorado Springs",
"AddressRegion": "CO",
"PostalCode": "80840",
"StreetAddress": "100 Main Street"
},
"BirthDate": "1979-10-12",
"Colleagues": [],
"Email": "info@example.com",
"Gender": "female",
"JobTitle": "Research Assistant",
"Name": "ERROR",
"Nationality": "Albanian",
"Relations": [
{
"$type": "TPersonRelation",
"Description": "Brother",
"Name": "John Doe"
},
{
"$type": "TPersonRelation",
"Description": "Mother",
"Name": "Mia Reyes"
}
],
"Telephone": "(123) 456-6789",
"URL": "http://www.example.com"
}
Going back in history
The TPerson object now contains a name with the value "ERROR". For the purpose of this blog post, this is simulated, but eventually in a real life example this could be a database read error, or an exception during the usage of the application which results in corrupt data. Now that we use the undo/redo manager and have saved an object state before changing the value to "ERROR", we can now revert back or "undo" the action that led into the error. simply call
which will go from the error state to the initial statevar
p: TPerson;
u: TTMSFNCUndoManager;
begin
p := TPerson.Create;
u := TTMSFNCUndoManager.Create(p);
try
TTMSFNCObjectPersistence.LoadObjectFromString(p, jsonSample);
u.PushState('init');
p.Name := 'ERROR';
u.PushState('error_data');
u.Undo;
p.Log;
finally
u.Free;
p.Free;
end;
end;
Going forward in history
Going forward in the object history stack, back to the state with the value "ERROR" as a name, we simply call