Solved

Tips on architecture for consuming/wrapping large webservice

Posted on 2012-03-12
7
209 Views
Last Modified: 2012-03-26
Hello Experts,

I am looking for good practices for implementing a smart architecture and way to handle integration against a system with many different wdsl webservices.

I have been hobby developing with C# for 2 years~, by that I am not always using the correct terminology, but I'll try to describe what I am looking for.

The main reason I'm posting this, is to get ideas of areas that I should read up on, design patterns to implement and how to manage API calls in a good way.

I'm integrating against a system that offers lots of different API's. Each API have 20-30 methods. Each API takes a number of parameters in XML format.

To give you an idea, I have taken the following API as an example, its the DeviceManager API and the Create method to create a device in the system.
The API takes two parameters, a string key and a XML parameter string.


Example

DeviceManager

public string Create(string sKey, string sXmlParameters)  

Parameters:

Name: sKey Type: string  
Description:
The API security key. A valid API security key is required for every API call.  

Name: sXmlParameters Type: string  
Description: Values needed to create a new device. See the Additional Information      section for XML format.  

Return Value: Type: string  
 Description: Returns XML containing the status of the creation and if the status is  
 Constants.ExternalServiceReturnCodes.SUCCEEDED, the ID of the new device.


XMLParameters:


<PARAMETERS>      
                          <NAME>  
                                   string - up to 64 characters. Non-blank name of the device. Must be  
                                   unique within a gateway.  
                          </NAME>  
                          <DESCRIPTION>  
                                   string - up to 1024 characters. The description of the new device.  
                          </DESCRIPTION> (optional)  
                          <TYPEID>  
                                   string of DeviceType. The type of device.
                          </TYPEID>  
                            <GATEWAYID>  
                                     string - 32-character GUID. The ID of the gateway to associate  
                                     with the device. If this node is included, it must contain an ID of  
                                     a gateway that exists in the solution.  
                            </GATEWAYID> (optional)  
                            <INSTALLATIONDATETIME>  
                                     date time in UTC - greater than or equal to  
                                     1753-01-01 00:00:00.000 and less than or equal to  
                                     9999- 12-31 23:59:59.998. The date time that the device was  
                                     installed.  
                            </INSTALLATIONDATETIME> (optional - if not included, the installation  
                            date time is set to the date time in UTC when the device is created in
                            the solution)      
                            <SERIALNUMBER>  
                                     string - up to 64 characters. The serial number of the device.  
                            </SERIALNUMBER>
                            <TIMEZONEID>  
                                     string - time zone ID. The time zone ID of the device. Call  
                                     the TimeZoneManager.RetrieveList API to get a list of  
                                     valid time zone IDs  
                            </TIMEZONEID> (required for device type 'meter')  
                            <HARDWAREVERSION>  
                                     string - up to 32 characters. The hardware version of the device.  
                            </HARDWAREVERSION> (optional)  
                            <GEOGRAPHICCOORDINATES>  
                                     <LATITUDE>  
                                               decimal - greater than or equal to -90 and less than or  
                                               equal to 90. The latitude geographical coordinate of the  
                                               device in decimal degrees. The value must be from the  
                                              WGS84 spatial reference system.  
                                               If more than 6 digits after the decimal point are included,  
                                               the value will be rounded to 6 digits.  
                                     </LATITUDE>  
                                     <LONGITUDE>  
                                               decimal - greater than or equal to -180 and less than or  
                                               equal to 180. The longitude geographical coordinate of the  
                                               device in decimal degrees. The value must be from the  
                                              WGS84 spatial reference system.
                                               If more than 6 digits after the decimal point are included,  
                                              the value will be rounded to 6 digits.  
                                     </LONGITUDE>  
                            </GEOGRAPHICCOORDINATES> (optional)
                            <METER>  
                                     <ID>  
                                               string - 12 hexadecimal characters.
                                     </ID>
                                     <TRANSFORMERID>  
                                              string - up to 128 characters.
                                     </TRANSFORMERID>  
                                     <DOWNLIMIT>  
                                                  integer - greater than or equal to 0 and less than or  
                                                  equal to 65535.
                                        </DOWNLIMIT> (optional)
                              </METER> (required for device type of 'meter')  
                    </PARAMETERS>

Return API payload format:


<DEVICEID>string - 32-character GUID. The ID of the new device.</DEVICEID>


The API's always return the data in the following format:


<RETURNS>
        <STATUS>
                 String from Constants.ExternalServiceReturnCodes class.
        </STATUS>
        <APIPAYLOAD>
                 Additional information from the API call. Node may be empty. For
                 APIs that return information, that information will be shown in
                 the Return section.
        </APIPAYLOAD>
</RETURNS>


So in the example above, the payload would look something like:


<RETURNS>
        <STATUS>
                 SUCCEEDED
        </STATUS>
        <APIPAYLOAD>
                <DEVICEID>string - 32-character GUID. The ID of the new device.</DEVICEID>
        </APIPAYLOAD>
</RETURNS>

As for now, I have been working on designing classes for all XMLparameters to be able to serialize and deserialize the parameters and payloads from the APIs, but it is a very time consuming job to define these. Example given below for the create API parameters and payload. (simple example using get set)

DeviceManager Create Parameters
 [XmlRoot(ElementName = "PARAMETERS")]
    public class Create
    {

        [XmlElement(ElementName = "NAME")]
        public string Name { get; set; }

        [XmlElement(ElementName = "DESCRIPTION")]
        public string Description { get; set; }

        [XmlElement(ElementName = "TYPEID")]
        public string TypeId { get; set; }

        [XmlElement(ElementName = "GATEWAYID")]
        public string GatewayId { get; set; }

        [XmlElement(ElementName = "INSTALLATIONDATETIME")]
        public string InstallationDateTime { get; set; }

        [XmlElement(ElementName = "SERIALNUMBER")]
        public string SerialNumber { get; set; }

        [XmlElement(ElementName = "TIMEZONEID")]
        public string TimeZoneId { get; set; }

        [XmlElement(ElementName = "HARDWAREVERSION")]
        public string HardWareVersion { get; set; }

        [XmlElement(ElementName = "GEOGRAPHICCOORDINATES")]
        public CreateParametersGeoGraphicCoordinates GeographicCoordinates { get; set; }

        [XmlElement(ElementName = "METER")]
        public CreateMeter Meter { get; set; }
    }

    public class CreateMeter
    {
        [XmlElement(ElementName = "NEURONID")]
        public string NeuronId { get; set; }

        [XmlElement(ElementName = "TRANSFORMERID")]
        public string TransformerId { get; set; }

        [XmlElement(ElementName = "UTILITYID")]
        public string UtilityId { get; set; }

        [XmlElement(ElementName = "LONTALKKEY")]
        public string LonTalkKey { get; set; }

        [XmlElement(ElementName = "DOWNLIMIT")]
        public string DownLimit { get; set; }
    }

    public class CreateParametersGeoGraphicCoordinates
    {
        [XmlElement(ElementName = "LATITUDE")]
        public string Latitude { get; set; }

        [XmlElement(ElementName = "LONGITUDE")]
        public string Longitude { get; set; }
    }

Open in new window


And for PayLoads I have the following generic class and DeviceManager.Create Payload specific class:

Create PayLoad
    public class CreatePayLoad
    {
        [XmlElement(ElementName = "DEVICEID")]
        public string DeviceId { get; set; }
    }

Open in new window


PayLoad
    [XmlRoot(ElementName = "RETURNS")]
    public class PayLoad<T> where T : new()
    {
        public PayLoad()
        {
            ApiPayLoad = new T();
        }

        /// <summary>
        /// Contains the payload from the command.
        /// </summary>
        [XmlElement(ElementName = "APIPAYLOAD")]
        public T ApiPayLoad { get; set; }

        /// <summary>
        /// Status of the call
        /// </summary>
        [XmlElement(ElementName = "STATUS")]
        public string Status { get; set; }
    }

Open in new window


So in my code i can do the call in the following way:

Example use

//Create the parameters
var parameters = new Create
					 {
						 Description = "Description",
						 GatewayId = "GatewayId",
						 Name = "NameOfDevice",
						 GeographicCoordinates = new CreateParametersGeoGraphicCoordinates
													 {
														 Latitude = "Lat",
														 Longitude = "Long"
													 },
						 Meter = new CreateMeter
									 {
										 TransformerId = "ID",
										 DownLimit = "120"
									 }
					 };

//Serialize the parameters to xml
string sXmlParameters = Helper.SerializeToXml<Create>(parameters);

//API call
string apiPayLoad = DeviceManager.Create(apiKey, sXmlParameters);

//Deserialize payload
var payLoad = Helper.DeserializeFromXml<PayLoad<CreatePayLoad>>(apiPayLoad);

}

Open in new window


Can someone please contribute with ideas and better ways to manage this?
Please bear in mind that there are about 300 methods in the system, some with very complicated xml parameters where some properties are optional, some mandatory if property A, B or C are used and so on.

I have been looking at XSD.exe, but the generated code is not neat to the eye and it doesn't handle collections very well.

A friend also proposed T4 templates, where one could generate classes based on templates, but I'm not really finding any good examples out there.

I'm not sure if I have explained myself in a good way, If I'm being unclear - please let me know and I will try to elaborate.

Thank you,
amr-it
0
Comment
Question by:amr-it
  • 5
  • 2
7 Comments
 
LVL 22

Expert Comment

by:ambience
ID: 37716234
There is no correct way of doing that and whatever suits and works is the way to go, but just to give an idea - if I were to do it I would probably also look at using a Fluent interface (kind of like a DSL).

var result = DeviceManager.Create(apiKey,
      Parameters.New()
      .Add("Name", "myName")
      .Add("typeid", "")
      .Add("geographiccoordinates",
                  Parameters.New()
                        .Add("latitude", 90)
                        .Add("longitude", 90)
            )
      .Add("timezoneid","")
      .ToXML()
);

var resultset = from response in Resultset.NewFromXML(result).Child("returns")
select new { Status = Resultset.ValueOf("Status") };

Internally, the Parameters would use XML and LINQ

http://www.programminghelp.com/programming/dotnet/using-linq-to-xml-to-add-data-to-xml-file-in-c/

Not to mention, you can have another layer of abstraction on top of that that would ease in building requests.

--

BTW, I havent tried that and dont think it would even compile but I just wanted to float an idea. The con to this approach is that its probably not as type safe as the traditional approach of mapping to POCO (what you are doing).

However, if you want to be able to pass around domain objects to other systems then its probably better to build a set of POCOs (contracts). This approach, however would shield your POCO from the intricacies of XML serialization and you can design domain objects fitting your style and not just a one-to-one mapping of the XML.
0
 
LVL 22

Expert Comment

by:ambience
ID: 37716254
BTW, this is even better (Imaging using extension methods to extend the basic DSL)

var result = DeviceManager.Create(apiKey,
      Parameters.
      .WithName("myName")
      .WithTypeId("")
      .With("geographiccoordinates",
                  Parameters.New()
                        .WithLatitude(90)
                        .WithLongitude(90)
            )
      .Validate()
      .asXML()
);

Validate() would be the place to check for presence of required parameters.
0
 
LVL 1

Author Comment

by:amr-it
ID: 37737621
Hi ambience,

I really like your suggestion on using DSL as that would help me with building the parameters as none of the APIs are strongly typed. I guess I could do the validation in a base class or so and just switch the parameter type.

var result = DeviceManager.Create(apiKey,
  Parameters.
  .Type(Parameters.Create)
  .Name("Name of Device")
  .Description("Description of Device")
  .Coordinates("Lat","Long")
  .Validate()
  .asXML()
);

Open in new window


However, the system I'm working with, have many different payloads, would you suggest to work in my current direction of mapping those payloads to objects using de-serialization? E.g having a service class and a business class that implements the same interface for casting?
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 22

Expert Comment

by:ambience
ID: 37737999
Im not sure I completely understood the idea of service and business class, so please elaborate that a bit.

>> However, the system I'm working with, have many different payloads

Question is, do you really need strongly typed business objects? A business can itself be a strongly typed DSL on a Hashtable for example?

How about standardizing on a DTO to represent hierarchical data effectively and then build a DSL that operates on top of that?

For example the Resultset DSL can be implemented on top of an Expando instead of XDocument, or even Dictionary.

// this snippet gets the Status from a Response
var resultset = from response in Resultset.FromXML(result).Returns
select new { Status = response.Status };

OR w/out LINQ

Resultset.FromXML(result).Returns.Status

whenever you need to pass around the data you can always do,

Resultset.FromXML(result).ToDictionary (there are a few other conveniece extension methods) http://msdn.microsoft.com/en-us/library/system.dynamic.expandoobject.aspx

or construct a strongly typed business object from that. I believe a little bit of Reflection can help you create an extension method where you can fill in an object's fields. To give an idea see the way reset() is implemented in this article

http://blogs.msdn.com/b/alexj/archive/2010/02/13/tip-54-how-to-improve-performance-using-statement-expressions.aspx

----

I hope you can make sense out of my incoherent collection of ideas.
0
 
LVL 22

Expert Comment

by:ambience
ID: 37738023
BTW, you can discard all the different ideas and just use the last link on Statement Expressions and add extension methods that can convert a business object to XML and vice/versa.

Of course that is when you must have business objects.
0
 
LVL 1

Author Comment

by:amr-it
ID: 37740997
Sorry I was a bit unclear in my previous post. What I ment was to have DTO classes for the different payloads. Many methods have similair returns, that's why I tried to make a generic payload class:

    [XmlRoot(ElementName = "RETURNS")]
    public class PayLoad<T> where T : new()
    {
        public PayLoad()
        {
            ApiPayLoad = new T();
        }

        /// <summary>
        /// Contains the payload from the command.
        /// </summary>
        [XmlElement(ElementName = "APIPAYLOAD")]
        public T ApiPayLoad { get; set; }

        /// <summary>
        /// Status of the call
        /// </summary>
        [XmlElement(ElementName = "STATUS")]
        public string Status { get; set; }
    }

Open in new window


Where T is the strongly typed DTO. However, due to my limited knowledge of design patterns, I'd appreciate input on how I could work with the payloads with code examples.

One example is the RetrieveList method of the DeviceManager:
public string RetrieveList(string sKey, string sXmlParameters)  

Open in new window


It takes a bunch of parameters which can be managed using DSL.
The response however is a bit complex (as many other payloads) and creating DTO's for all these is a very time consuming task, but might be the best thing to do?
However, I have also come across the statement that using DTO is an anti-pattern as you will have duplicate DO's?

The funny thing is, that the Device entity in the Devices list, is not necessary the same as a device returned from a different method :(
Do you have any tips on how to handle this in a smart way? I will most likely be calling different API's and perform additional calls with the data retrieved in
the first payload. E.g. calling DeviceManager.RetrieveList - selecting a list of devices with a specific search criteria and then Update them or do something else with them.

Payload:
<RETURNS>
		<STATUS>Status of Call</STATUS>
		<APIPAYLOAD>
                   <COUNT>  
                             integer - the total number of devices matching the search criteria 
                   </COUNT>  
                   <DEVICES>          
                             <DEVICE>              
                                      <ID>string - 32-character GUID. The device ID.</ID>    
                                      <NAME>string - up to 64 characters. The device name.</NAME>  
                                      <TYPEID>  
                                                string from Constants.DeviceTypes class. The device type.  
                                      </TYPEID>  
                                      <STATUSTYPEID>  
												The status of the device  
                                      </STATUSTYPEID>  
                                      <SERIALNUMBER>  
                                                string - up to 64 characters. The device serial number.  
                                      </SERIALNUMBER>  
                                      <LASTGATEWAYTODEVICECOMMUNICATIONSTATUSTYPEID>  
                                               The status of the last communication from the gateway to the device.  
                                      </LASTGATEWAYTODEVICECOMMUNICATIONSTATUSTYPEID> 
                                      <METER>  
                                                <STATUSTYPEID>  
                                                         string from Constants.DeviceLoadVoltageStatusTypes   
                                                         class. The status of the load voltage on the meter.  
                                                </STATUSTYPEID>  
                                                <FEEDBACKSTATUSTYPE>  
                                                        Status of the feedback on the meter.  
                                               </FEEDBACKSTATUSTYPE>  
                                               <ID>  
														The ID.            
                                               </ID>  
                                               <SUBNET> 
														Subnet of Device
                                               </SUBNET>  
                                               <NODE>  
                                                        Node of Device 
                                               </NODE>  
                                               <TOTALSIGNALSTRENGTH>  
                                                        Signal Strength 
                                               </TOTALSIGNALSTRENGTH>  
                                               <CM>  
                                                        CM  
                                               </CM>  
                                               <REPEATERCOUNT>  
                                                        RC  
                                               </REPEATERCOUNT>  
                                               <LASTRECEIVEDRESULTS>  
                                                        <LASTRECEIVEDRESULT>  
                                                                 <RESULTTYPEID>  
                                                                           ResultType  
                                                                 </RESULTTYPEID>  
                                                                 <DATETIME>  
                                                                           dt 
                                                                   </DATETIME>  
                                                         </LASTRECEIVEDRESULT>  
                                                         .. for each result type 
                                                </LASTRECEIVEDRESULTS>  
                                      </METER>  
									  <GATEWAY>  
                                                  <ID>  
                                                           string - 32-character GUID. The ID of the gateway   
                                                           the device is associated with.  
                                                  </ID>  
                                                  <NAME>  
                                                           string - up to 64 characters. The name of the gateway   
                                                           the device is associated with.  
                                                 </NAME>  
                                                 <SERIALNUMBER>  
                                                           string - up to 64 characters. The serial number of the   
                                                           gateway the device is associated with.  
                                                 </SERIALNUMBER>  
                                                 <STATUSTYPEID>  
                                                           Status of gateway 
                                                 </STATUSTYPEID>  
                                        </GATEWAY>
                                        <PARENTDEVICE>  
                                                 <ID>  
                                                           string - 32-character GUID. The ID of the device's   
                                                           parent device.  
                                                 </ID>       
                                       </PARENTDEVICE>  
                              </DEVICE>  
                              ...  for each device
                    </DEVICES>   
                    <DATASET>  
                              <NEXT>  
                                     <ADDITIONAL>  
                                         Indicates if there are additional devices to be fetched to the resultset  
                                     </ADDITIONAL>  
                                     <DEVICEID>  
                                         DeviceID to pass for next call (paging) 
                                     </DEVICEID>  
							</NEXT>  
                     </DATASET>
			</APIPAYLOAD>
</RETURNS>

Open in new window

0
 
LVL 22

Accepted Solution

by:
ambience earned 500 total points
ID: 37760638
I got late, sorry for that

>> Do you have any tips on how to handle this in a smart way?

Smart is a pretty relative thing and probably no end of smartness :)

- Just use LINQ and XML and fetch desired values from response XML. Not great, unless the use of response is localized and you dont need to pass around the exact response.

- Use domain objects, as many as needed. Works across component boundaries, strongly typed. Lot of work, but no repetition. If you match the schema then just deserialize XML into response -- this is what you are already doing.

- Parse the response into a DTO and build DSL to access it, similar to what you did for the Request Parameters. Possibly repetitive use of DSL whenever you need to use the DTO. The good is that you need not have 300 classes for different payloads. (NOTE: there is no business object here).

Probably another level of smartness would be that whatever approach you take, you can always build a DSL to make it easy to work with and factor out common functionality into reusable language constructs.

I would love to hear other ideas, because at this moment I can't think of any other way.


-
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Article by: Najam
Having new technologies does not mean they will completely replace old components.  Recently I had to create WCF that will be called by VB6 component.  Here I will describe what steps one should follow while doing so, please feel free to post any qu…
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now