Link to home
Start Free TrialLog in
Avatar of amr-it
amr-it

asked on

Tips on architecture for consuming/wrapping large webservice

Hello Experts,

I am looking for good practices for implementing a smart architecture and way to handle integration against a system with many different wdsl webservices.

I have been hobby developing with C# for 2 years~, by that I am not always using the correct terminology, but I'll try to describe what I am looking for.

The main reason I'm posting this, is to get ideas of areas that I should read up on, design patterns to implement and how to manage API calls in a good way.

I'm integrating against a system that offers lots of different API's. Each API have 20-30 methods. Each API takes a number of parameters in XML format.

To give you an idea, I have taken the following API as an example, its the DeviceManager API and the Create method to create a device in the system.
The API takes two parameters, a string key and a XML parameter string.


Example

DeviceManager

public string Create(string sKey, string sXmlParameters)  

Parameters:

Name: sKey Type: string  
Description:
The API security key. A valid API security key is required for every API call.  

Name: sXmlParameters Type: string  
Description: Values needed to create a new device. See the Additional Information      section for XML format.  

Return Value: Type: string  
 Description: Returns XML containing the status of the creation and if the status is  
 Constants.ExternalServiceReturnCodes.SUCCEEDED, the ID of the new device.


XMLParameters:


<PARAMETERS>      
                          <NAME>  
                                   string - up to 64 characters. Non-blank name of the device. Must be  
                                   unique within a gateway.  
                          </NAME>  
                          <DESCRIPTION>  
                                   string - up to 1024 characters. The description of the new device.  
                          </DESCRIPTION> (optional)  
                          <TYPEID>  
                                   string of DeviceType. The type of device.
                          </TYPEID>  
                            <GATEWAYID>  
                                     string - 32-character GUID. The ID of the gateway to associate  
                                     with the device. If this node is included, it must contain an ID of  
                                     a gateway that exists in the solution.  
                            </GATEWAYID> (optional)  
                            <INSTALLATIONDATETIME>  
                                     date time in UTC - greater than or equal to  
                                     1753-01-01 00:00:00.000 and less than or equal to  
                                     9999- 12-31 23:59:59.998. The date time that the device was  
                                     installed.  
                            </INSTALLATIONDATETIME> (optional - if not included, the installation  
                            date time is set to the date time in UTC when the device is created in
                            the solution)      
                            <SERIALNUMBER>  
                                     string - up to 64 characters. The serial number of the device.  
                            </SERIALNUMBER>
                            <TIMEZONEID>  
                                     string - time zone ID. The time zone ID of the device. Call  
                                     the TimeZoneManager.RetrieveList API to get a list of  
                                     valid time zone IDs  
                            </TIMEZONEID> (required for device type 'meter')  
                            <HARDWAREVERSION>  
                                     string - up to 32 characters. The hardware version of the device.  
                            </HARDWAREVERSION> (optional)  
                            <GEOGRAPHICCOORDINATES>  
                                     <LATITUDE>  
                                               decimal - greater than or equal to -90 and less than or  
                                               equal to 90. The latitude geographical coordinate of the  
                                               device in decimal degrees. The value must be from the  
                                              WGS84 spatial reference system.  
                                               If more than 6 digits after the decimal point are included,  
                                               the value will be rounded to 6 digits.  
                                     </LATITUDE>  
                                     <LONGITUDE>  
                                               decimal - greater than or equal to -180 and less than or  
                                               equal to 180. The longitude geographical coordinate of the  
                                               device in decimal degrees. The value must be from the  
                                              WGS84 spatial reference system.
                                               If more than 6 digits after the decimal point are included,  
                                              the value will be rounded to 6 digits.  
                                     </LONGITUDE>  
                            </GEOGRAPHICCOORDINATES> (optional)
                            <METER>  
                                     <ID>  
                                               string - 12 hexadecimal characters.
                                     </ID>
                                     <TRANSFORMERID>  
                                              string - up to 128 characters.
                                     </TRANSFORMERID>  
                                     <DOWNLIMIT>  
                                                  integer - greater than or equal to 0 and less than or  
                                                  equal to 65535.
                                        </DOWNLIMIT> (optional)
                              </METER> (required for device type of 'meter')  
                    </PARAMETERS>

Return API payload format:


<DEVICEID>string - 32-character GUID. The ID of the new device.</DEVICEID>


The API's always return the data in the following format:


<RETURNS>
        <STATUS>
                 String from Constants.ExternalServiceReturnCodes class.
        </STATUS>
        <APIPAYLOAD>
                 Additional information from the API call. Node may be empty. For
                 APIs that return information, that information will be shown in
                 the Return section.
        </APIPAYLOAD>
</RETURNS>


So in the example above, the payload would look something like:


<RETURNS>
        <STATUS>
                 SUCCEEDED
        </STATUS>
        <APIPAYLOAD>
                <DEVICEID>string - 32-character GUID. The ID of the new device.</DEVICEID>
        </APIPAYLOAD>
</RETURNS>

As for now, I have been working on designing classes for all XMLparameters to be able to serialize and deserialize the parameters and payloads from the APIs, but it is a very time consuming job to define these. Example given below for the create API parameters and payload. (simple example using get set)

DeviceManager Create Parameters
 [XmlRoot(ElementName = "PARAMETERS")]
    public class Create
    {

        [XmlElement(ElementName = "NAME")]
        public string Name { get; set; }

        [XmlElement(ElementName = "DESCRIPTION")]
        public string Description { get; set; }

        [XmlElement(ElementName = "TYPEID")]
        public string TypeId { get; set; }

        [XmlElement(ElementName = "GATEWAYID")]
        public string GatewayId { get; set; }

        [XmlElement(ElementName = "INSTALLATIONDATETIME")]
        public string InstallationDateTime { get; set; }

        [XmlElement(ElementName = "SERIALNUMBER")]
        public string SerialNumber { get; set; }

        [XmlElement(ElementName = "TIMEZONEID")]
        public string TimeZoneId { get; set; }

        [XmlElement(ElementName = "HARDWAREVERSION")]
        public string HardWareVersion { get; set; }

        [XmlElement(ElementName = "GEOGRAPHICCOORDINATES")]
        public CreateParametersGeoGraphicCoordinates GeographicCoordinates { get; set; }

        [XmlElement(ElementName = "METER")]
        public CreateMeter Meter { get; set; }
    }

    public class CreateMeter
    {
        [XmlElement(ElementName = "NEURONID")]
        public string NeuronId { get; set; }

        [XmlElement(ElementName = "TRANSFORMERID")]
        public string TransformerId { get; set; }

        [XmlElement(ElementName = "UTILITYID")]
        public string UtilityId { get; set; }

        [XmlElement(ElementName = "LONTALKKEY")]
        public string LonTalkKey { get; set; }

        [XmlElement(ElementName = "DOWNLIMIT")]
        public string DownLimit { get; set; }
    }

    public class CreateParametersGeoGraphicCoordinates
    {
        [XmlElement(ElementName = "LATITUDE")]
        public string Latitude { get; set; }

        [XmlElement(ElementName = "LONGITUDE")]
        public string Longitude { get; set; }
    }

Open in new window


And for PayLoads I have the following generic class and DeviceManager.Create Payload specific class:

Create PayLoad
    public class CreatePayLoad
    {
        [XmlElement(ElementName = "DEVICEID")]
        public string DeviceId { get; set; }
    }

Open in new window


PayLoad
    [XmlRoot(ElementName = "RETURNS")]
    public class PayLoad<T> where T : new()
    {
        public PayLoad()
        {
            ApiPayLoad = new T();
        }

        /// <summary>
        /// Contains the payload from the command.
        /// </summary>
        [XmlElement(ElementName = "APIPAYLOAD")]
        public T ApiPayLoad { get; set; }

        /// <summary>
        /// Status of the call
        /// </summary>
        [XmlElement(ElementName = "STATUS")]
        public string Status { get; set; }
    }

Open in new window


So in my code i can do the call in the following way:

Example use

//Create the parameters
var parameters = new Create
					 {
						 Description = "Description",
						 GatewayId = "GatewayId",
						 Name = "NameOfDevice",
						 GeographicCoordinates = new CreateParametersGeoGraphicCoordinates
													 {
														 Latitude = "Lat",
														 Longitude = "Long"
													 },
						 Meter = new CreateMeter
									 {
										 TransformerId = "ID",
										 DownLimit = "120"
									 }
					 };

//Serialize the parameters to xml
string sXmlParameters = Helper.SerializeToXml<Create>(parameters);

//API call
string apiPayLoad = DeviceManager.Create(apiKey, sXmlParameters);

//Deserialize payload
var payLoad = Helper.DeserializeFromXml<PayLoad<CreatePayLoad>>(apiPayLoad);

}

Open in new window


Can someone please contribute with ideas and better ways to manage this?
Please bear in mind that there are about 300 methods in the system, some with very complicated xml parameters where some properties are optional, some mandatory if property A, B or C are used and so on.

I have been looking at XSD.exe, but the generated code is not neat to the eye and it doesn't handle collections very well.

A friend also proposed T4 templates, where one could generate classes based on templates, but I'm not really finding any good examples out there.

I'm not sure if I have explained myself in a good way, If I'm being unclear - please let me know and I will try to elaborate.

Thank you,
amr-it
Avatar of ambience
ambience
Flag of Pakistan image

There is no correct way of doing that and whatever suits and works is the way to go, but just to give an idea - if I were to do it I would probably also look at using a Fluent interface (kind of like a DSL).

var result = DeviceManager.Create(apiKey,
      Parameters.New()
      .Add("Name", "myName")
      .Add("typeid", "")
      .Add("geographiccoordinates",
                  Parameters.New()
                        .Add("latitude", 90)
                        .Add("longitude", 90)
            )
      .Add("timezoneid","")
      .ToXML()
);

var resultset = from response in Resultset.NewFromXML(result).Child("returns")
select new { Status = Resultset.ValueOf("Status") };

Internally, the Parameters would use XML and LINQ

http://www.programminghelp.com/programming/dotnet/using-linq-to-xml-to-add-data-to-xml-file-in-c/

Not to mention, you can have another layer of abstraction on top of that that would ease in building requests.

--

BTW, I havent tried that and dont think it would even compile but I just wanted to float an idea. The con to this approach is that its probably not as type safe as the traditional approach of mapping to POCO (what you are doing).

However, if you want to be able to pass around domain objects to other systems then its probably better to build a set of POCOs (contracts). This approach, however would shield your POCO from the intricacies of XML serialization and you can design domain objects fitting your style and not just a one-to-one mapping of the XML.
BTW, this is even better (Imaging using extension methods to extend the basic DSL)

var result = DeviceManager.Create(apiKey,
      Parameters.
      .WithName("myName")
      .WithTypeId("")
      .With("geographiccoordinates",
                  Parameters.New()
                        .WithLatitude(90)
                        .WithLongitude(90)
            )
      .Validate()
      .asXML()
);

Validate() would be the place to check for presence of required parameters.
Avatar of amr-it
amr-it

ASKER

Hi ambience,

I really like your suggestion on using DSL as that would help me with building the parameters as none of the APIs are strongly typed. I guess I could do the validation in a base class or so and just switch the parameter type.

var result = DeviceManager.Create(apiKey,
  Parameters.
  .Type(Parameters.Create)
  .Name("Name of Device")
  .Description("Description of Device")
  .Coordinates("Lat","Long")
  .Validate()
  .asXML()
);

Open in new window


However, the system I'm working with, have many different payloads, would you suggest to work in my current direction of mapping those payloads to objects using de-serialization? E.g having a service class and a business class that implements the same interface for casting?
Im not sure I completely understood the idea of service and business class, so please elaborate that a bit.

>> However, the system I'm working with, have many different payloads

Question is, do you really need strongly typed business objects? A business can itself be a strongly typed DSL on a Hashtable for example?

How about standardizing on a DTO to represent hierarchical data effectively and then build a DSL that operates on top of that?

For example the Resultset DSL can be implemented on top of an Expando instead of XDocument, or even Dictionary.

// this snippet gets the Status from a Response
var resultset = from response in Resultset.FromXML(result).Returns
select new { Status = response.Status };

OR w/out LINQ

Resultset.FromXML(result).Returns.Status

whenever you need to pass around the data you can always do,

Resultset.FromXML(result).ToDictionary (there are a few other conveniece extension methods) http://msdn.microsoft.com/en-us/library/system.dynamic.expandoobject.aspx

or construct a strongly typed business object from that. I believe a little bit of Reflection can help you create an extension method where you can fill in an object's fields. To give an idea see the way reset() is implemented in this article

http://blogs.msdn.com/b/alexj/archive/2010/02/13/tip-54-how-to-improve-performance-using-statement-expressions.aspx

----

I hope you can make sense out of my incoherent collection of ideas.
BTW, you can discard all the different ideas and just use the last link on Statement Expressions and add extension methods that can convert a business object to XML and vice/versa.

Of course that is when you must have business objects.
Avatar of amr-it

ASKER

Sorry I was a bit unclear in my previous post. What I ment was to have DTO classes for the different payloads. Many methods have similair returns, that's why I tried to make a generic payload class:

    [XmlRoot(ElementName = "RETURNS")]
    public class PayLoad<T> where T : new()
    {
        public PayLoad()
        {
            ApiPayLoad = new T();
        }

        /// <summary>
        /// Contains the payload from the command.
        /// </summary>
        [XmlElement(ElementName = "APIPAYLOAD")]
        public T ApiPayLoad { get; set; }

        /// <summary>
        /// Status of the call
        /// </summary>
        [XmlElement(ElementName = "STATUS")]
        public string Status { get; set; }
    }

Open in new window


Where T is the strongly typed DTO. However, due to my limited knowledge of design patterns, I'd appreciate input on how I could work with the payloads with code examples.

One example is the RetrieveList method of the DeviceManager:
public string RetrieveList(string sKey, string sXmlParameters)  

Open in new window


It takes a bunch of parameters which can be managed using DSL.
The response however is a bit complex (as many other payloads) and creating DTO's for all these is a very time consuming task, but might be the best thing to do?
However, I have also come across the statement that using DTO is an anti-pattern as you will have duplicate DO's?

The funny thing is, that the Device entity in the Devices list, is not necessary the same as a device returned from a different method :(
Do you have any tips on how to handle this in a smart way? I will most likely be calling different API's and perform additional calls with the data retrieved in
the first payload. E.g. calling DeviceManager.RetrieveList - selecting a list of devices with a specific search criteria and then Update them or do something else with them.

Payload:
<RETURNS>
		<STATUS>Status of Call</STATUS>
		<APIPAYLOAD>
                   <COUNT>  
                             integer - the total number of devices matching the search criteria 
                   </COUNT>  
                   <DEVICES>          
                             <DEVICE>              
                                      <ID>string - 32-character GUID. The device ID.</ID>    
                                      <NAME>string - up to 64 characters. The device name.</NAME>  
                                      <TYPEID>  
                                                string from Constants.DeviceTypes class. The device type.  
                                      </TYPEID>  
                                      <STATUSTYPEID>  
												The status of the device  
                                      </STATUSTYPEID>  
                                      <SERIALNUMBER>  
                                                string - up to 64 characters. The device serial number.  
                                      </SERIALNUMBER>  
                                      <LASTGATEWAYTODEVICECOMMUNICATIONSTATUSTYPEID>  
                                               The status of the last communication from the gateway to the device.  
                                      </LASTGATEWAYTODEVICECOMMUNICATIONSTATUSTYPEID> 
                                      <METER>  
                                                <STATUSTYPEID>  
                                                         string from Constants.DeviceLoadVoltageStatusTypes   
                                                         class. The status of the load voltage on the meter.  
                                                </STATUSTYPEID>  
                                                <FEEDBACKSTATUSTYPE>  
                                                        Status of the feedback on the meter.  
                                               </FEEDBACKSTATUSTYPE>  
                                               <ID>  
														The ID.            
                                               </ID>  
                                               <SUBNET> 
														Subnet of Device
                                               </SUBNET>  
                                               <NODE>  
                                                        Node of Device 
                                               </NODE>  
                                               <TOTALSIGNALSTRENGTH>  
                                                        Signal Strength 
                                               </TOTALSIGNALSTRENGTH>  
                                               <CM>  
                                                        CM  
                                               </CM>  
                                               <REPEATERCOUNT>  
                                                        RC  
                                               </REPEATERCOUNT>  
                                               <LASTRECEIVEDRESULTS>  
                                                        <LASTRECEIVEDRESULT>  
                                                                 <RESULTTYPEID>  
                                                                           ResultType  
                                                                 </RESULTTYPEID>  
                                                                 <DATETIME>  
                                                                           dt 
                                                                   </DATETIME>  
                                                         </LASTRECEIVEDRESULT>  
                                                         .. for each result type 
                                                </LASTRECEIVEDRESULTS>  
                                      </METER>  
									  <GATEWAY>  
                                                  <ID>  
                                                           string - 32-character GUID. The ID of the gateway   
                                                           the device is associated with.  
                                                  </ID>  
                                                  <NAME>  
                                                           string - up to 64 characters. The name of the gateway   
                                                           the device is associated with.  
                                                 </NAME>  
                                                 <SERIALNUMBER>  
                                                           string - up to 64 characters. The serial number of the   
                                                           gateway the device is associated with.  
                                                 </SERIALNUMBER>  
                                                 <STATUSTYPEID>  
                                                           Status of gateway 
                                                 </STATUSTYPEID>  
                                        </GATEWAY>
                                        <PARENTDEVICE>  
                                                 <ID>  
                                                           string - 32-character GUID. The ID of the device's   
                                                           parent device.  
                                                 </ID>       
                                       </PARENTDEVICE>  
                              </DEVICE>  
                              ...  for each device
                    </DEVICES>   
                    <DATASET>  
                              <NEXT>  
                                     <ADDITIONAL>  
                                         Indicates if there are additional devices to be fetched to the resultset  
                                     </ADDITIONAL>  
                                     <DEVICEID>  
                                         DeviceID to pass for next call (paging) 
                                     </DEVICEID>  
							</NEXT>  
                     </DATASET>
			</APIPAYLOAD>
</RETURNS>

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of ambience
ambience
Flag of Pakistan image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial