Andy
asked on
Retrieving JSON via REST Interface
I am trying to create a method which can retrieve JSON objects. I’m using the Jama API (https://dev.jamasoftware.com/rest)
To simplify it I am using RestSharp
I start by using the Jama swagger tool to test the interface. This produces a Response Body. I take this response body and in visual studio create a new class using the paste special ‘paste JSON as classes’ command.
This results in a new class:
I have then attempted to craft a new method which can retrieve all the records (the max number of results is limited to 50)
Now this works but is very slow, I’ve also had trouble in returning the generic list, I’ve got it working by returning an object, but then I need to mess about by casting it back into the correct type.
For record I call it like this:
I’m no expert in this (which is evident in this code I expect) I’d like to make it faster and also fix the retuning issue.
The important thing here is that the getAllRecords method should be able to process different classes.
Appreciate all help.
To simplify it I am using RestSharp
I start by using the Jama swagger tool to test the interface. This produces a Response Body. I take this response body and in visual studio create a new class using the paste special ‘paste JSON as classes’ command.
This results in a new class:
public class Item
{
public class Rootobject
{
public Meta meta { get; set; }
public Links links { get; set; }
public Datum[] data { get; set; }
public static explicit operator Rootobject(List<object> v)
{
throw new NotImplementedException();
}
}
public class Meta
{
public string status { get; set; }
public DateTime timestamp { get; set; }
public Pageinfo pageInfo { get; set; }
}
public class Pageinfo
{
public int startIndex { get; set; }
public int resultCount { get; set; }
public int totalResults { get; set; }
}
public class Links
{
public DataFieldsLookup5 datafieldslookup5 { get; set; }
public DataLockLockedby datalocklockedBy { get; set; }
public DataFieldsLookup6 datafieldslookup6 { get; set; }
public DataFieldsLookup3 datafieldslookup3 { get; set; }
public DataFieldsLookup4 datafieldslookup4 { get; set; }
public DataModifiedby datamodifiedBy { get; set; }
public DataFieldsStatus datafieldsstatus { get; set; }
public DataFieldsLookup7 datafieldslookup7 { get; set; }
public DataLocationParentItem datalocationparentitem { get; set; }
public DataFieldsLookup1 datafieldslookup1 { get; set; }
public DataFieldsLookup2 datafieldslookup2 { get; set; }
public DataCreatedby datacreatedBy { get; set; }
public DataChilditemtype datachildItemType { get; set; }
public DataProject dataproject { get; set; }
public DataItemtype dataitemType { get; set; }
public DataLocationParentProject datalocationparentproject { get; set; }
}
public class DataFieldsLookup5
{
public string type { get; set; }
public string href { get; set; }
}
public class DataLockLockedby
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup6
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup3
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup4
{
public string type { get; set; }
public string href { get; set; }
}
public class DataModifiedby
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsStatus
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup7
{
public string type { get; set; }
public string href { get; set; }
}
public class DataLocationParentItem
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup1
{
public string type { get; set; }
public string href { get; set; }
}
public class DataFieldsLookup2
{
public string type { get; set; }
public string href { get; set; }
}
public class DataCreatedby
{
public string type { get; set; }
public string href { get; set; }
}
public class DataChilditemtype
{
public string type { get; set; }
public string href { get; set; }
}
public class DataProject
{
public string type { get; set; }
public string href { get; set; }
}
public class DataItemtype
{
public string type { get; set; }
public string href { get; set; }
}
public class DataLocationParentProject
{
public string type { get; set; }
public string href { get; set; }
}
public class Datum
{
public int id { get; set; }
public string documentKey { get; set; }
public string globalId { get; set; }
public int itemType { get; set; }
public int project { get; set; }
public DateTime createdDate { get; set; }
public DateTime modifiedDate { get; set; }
public DateTime lastActivityDate { get; set; }
public int createdBy { get; set; }
public int modifiedBy { get; set; }
public Fields fields { get; set; }
public Resources resources { get; set; }
public Location location { get; set; }
public Lock _lock { get; set; }
public string type { get; set; }
public int childItemType { get; set; }
}
public class Fields
{
public string documentKey { get; set; }
public string globalId { get; set; }
public string name { get; set; }
public string description { get; set; }
public int status { get; set; }
public string setKey { get; set; }
public int lookup1 { get; set; }
public int lookup2 { get; set; }
public string string1 { get; set; }
public int lookup3 { get; set; }
public int lookup4 { get; set; }
public int lookup5 { get; set; }
public int lookup6 { get; set; }
public int lookup7 { get; set; }
}
public class Resources
{
public Self self { get; set; }
}
public class Self
{
public string[] allowed { get; set; }
}
public class Location
{
public Parent parent { get; set; }
public int sortOrder { get; set; }
public int globalSortOrder { get; set; }
public string sequence { get; set; }
}
public class Parent
{
public int item { get; set; }
public int project { get; set; }
}
public class Lock
{
public bool locked { get; set; }
public DateTime lastLockedDate { get; set; }
public int lockedBy { get; set; }
}
}I have then attempted to craft a new method which can retrieve all the records (the max number of results is limited to 50)
public object getAllRecords<T>(List<T> genericList, string RestCommand, string[,] parameters, string [,] urlSegments)
{
int pageCount = 50;
var client = new RestClient("https://*******");
client.Authenticator = new HttpBasicAuthenticator("*******", "*****");
var request = new RestRequest(RestCommand, Method.GET);
// add the parameters
if (parameters.Length > 0)
{
for (int i = 0; i < parameters.Length / 2; i++)
{
request.AddParameter(parameters[i, 0], parameters[i, 1]);
}
}
if (urlSegments.Length > 0)
{
for (int i = 0; i < urlSegments.Length / 2; i++)
{
request.AddUrlSegment(urlSegments[i, 0], urlSegments[i, 1]);
}
}
var response = client.Execute(request);
var content = response.Content;
if (typeof(T) == typeof(Rootobject))
{
List<Rootobject> tempObject = new List<Rootobject>();
tempObject.Cast<Rootobject>();
Rootobject myRel = JsonConvert.DeserializeObject<Rootobject>(content);
tempObject.Add(myRel);
// create logic for looping through remaining records
int totalRecords = myRel.meta.pageInfo.totalResults;
do
{
request.AddParameter("startAt", pageCount);
response = client.Execute(request);
content = response.Content;
Rootobject myRel1 = JsonConvert.DeserializeObject<Rootobject>(content);
tempObject.Add(myRel1);
pageCount += 50;
} while (pageCount <= totalRecords);
List<Item.Datum> totalList = new List<Item.Datum>();
foreach (Rootobject rootItem in tempObject)
{
foreach (Item.Datum item in rootItem.data)
{
totalList.Add(item);
}
}
return totalList as List<Item.Datum>;
}
else if (typeof(T) == typeof(rootDownstreamrelated))
{
List<rootDownstreamrelated> tempObject = new List<rootDownstreamrelated>();
tempObject.Cast<rootDownstreamrelated>();
rootDownstreamrelated myRel = JsonConvert.DeserializeObject<rootDownstreamrelated>(content);
tempObject.Add(myRel);
// create logic for looping through remaining records
int totalRecords = myRel.meta.pageInfo.totalResults;
do
{
request.AddParameter("startAt", pageCount);
response = client.Execute(request);
content = response.Content;
rootDownstreamrelated myRel1 = JsonConvert.DeserializeObject<rootDownstreamrelated>(content);
tempObject.Add(myRel1);
pageCount += 50;
} while (pageCount <= totalRecords);
List<downstreamrelated.Datum> totalList = new List<downstreamrelated.Datum>();
foreach (rootDownstreamrelated rootItem in tempObject)
{
foreach (downstreamrelated.Datum item in rootItem.data)
{
totalList.Add(item);
}
}
return totalList as List<downstreamrelated.Datum>;
}
return null;
}Now this works but is very slow, I’ve also had trouble in returning the generic list, I’ve got it working by returning an object, but then I need to mess about by casting it back into the correct type.
For record I call it like this:
var allItems = getAllRecords(new List<Rootobject>(), "abstractitems", new string[,] { { "project", "1" }, { "itemType", "141" },
{ "maxResults", "50" } }, new string[,] { });
List<Item.Datum> objectList = allItems as List<Item.Datum>;I’m no expert in this (which is evident in this code I expect) I’d like to make it faster and also fix the retuning issue.
The important thing here is that the getAllRecords method should be able to process different classes.
Appreciate all help.
ASKER
The slowness is when it reaches the
If I try the swagger page it'll retrieve in an instant. So the delay is not with the de-serialisation portion.
You mentioned a new approach, I think that is beyond my current skill level, are you able to help me walk through an example of how it could be achieved?
client.ExecuteIf I try the swagger page it'll retrieve in an instant. So the delay is not with the de-serialisation portion.
You mentioned a new approach, I think that is beyond my current skill level, are you able to help me walk through an example of how it could be achieved?
[..] this takes about 6 seconds to complete, so to recall 1500 records, means about 50 executes and total time of 3 minutes. This is very long.The trick is simple: reduce the number of calls. When not possible parallelize them. But I cannot test both for obvious reasons.
Did you already ask the vendor, whether they offer ready to use client libraries?
When I said pretty complex, I meant that. It normally takes some weeks to implement this correct.
But in your case, as you said you're no export in this (btw, do you refer to programming or REST API's), start with some simple use-cases. Implement them as concrete cases. Don't start with abstractions, cause the method signature already rises some questions:
public object getAllRecords<T>(List<T> genericList, string RestCommand, string[,] parameters, string [,] urlSegments)- Where are the constraints for T?
- Normally you would return IEnumerable instead of List.
- RestCommand should be a type. At least an enum, but often enough it should be a class. Either as strategy template or command template.
- C# is a strongly typed language. Having [,] string arrays in many cases is a bad idea.
Things which come into mind for cleaner code using abstraction:
1) use a factory method to create the client and let it also handle the authentication. This can be a method in an abstract base class.
2) Take a look at the REST API. It is often a good idea to encapsulate execute calls also get a unified exception handling and especially exception logging.
3) Take a look at the REST API. Often pagination is conformed. This means that it can be also abstracted.
4) Consider using factory methods or builder classes to create your data classes.
This question needs an answer!
Become an EE member today
7 DAY FREE TRIALMembers can start a 7-Day Free trial then enjoy unlimited access to the platform.
View membership options
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Well, this is a pretty complex task. Cause it sounds like you want/need a client library.
In this case you need to abstract what that Jama API delivers. Thus simplifying it by using RestSharp is going the way in the opposite direction.
You need to look at network/transport layer and the response time. Often this is caused by the speed of transmission.