Could you check compliance, suitability of this kind of JSON via data exchange API?

Eduardo Fuerte
Eduardo Fuerte used Ask the Experts™
on
Hi Experts

Could you check compliance, suitability of this kind of JSON via data exchange API?

A page developed in PHP triggered by an operating system chron (Linux Chron) from time to time to obtain a JSON file saved on FTP server by the counterparty with the information to be collected.

A page developed in PHP triggered by an operating system chron (Linux Chron) from time to time for the generation of a JSON file saved on FTP server by the counterparty with the information to be collected.

Any better alternative to do this task?

Thanks in advance
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Describe...

1) API's purpose

2) Type/Volume of returned data payloads

3) Frequency of consumers/clients calling API

4) Effect of API calls on consumers/clients money/income/monetization

For example, if an API call is made a few times a week + has no bearing on anyone's money (so its a hobby), then you can do anything you like.

If your API has to sustain 1,000,000s calls/minute + involves $1,000,000s/month in revenue across 1,000s of consumers/clients, then best you hire an API developer, as API development is a fair bit different than Application or Website development.
Fractional CTO
Distinguished Expert 2018
Commented:
Tip: When I design an API, my criteria are...

1) >1,000,000 request/minute throughput.

2) Near zero disk I/O burden on API server side.

3) API calls occur via HTTPS + HTTP2 + Brotli using some sort of authentication key.

All 3x of these criteria are easily designed in from day one + near impossible to fix later on in time.

Also all 3x of these criteria allow for any type of API call volume to be handled very easily.

Note: There's also a 4th item I add in for extremely high volume API calls, which is a client side cache, so... poorly crafted consumer/client code leverages a client side cache, rather than hammering the server.
Eduardo FuerteDeveloper and Analyst

Author

Commented:
David

Presently it's still a draft.

We are planning some different kinds of API(s)

Data upload of residential properties / images/ banners/  and house services data.

I guess nothing to heavy or frequently. Still in survey's phase.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
I see mentioning of FTP... Strike that one from your list. replace with something sustainable: if you need file transfer: use SCP / SFTP.. (based on SSH).
Or make it part of your API.

When building an interface for data, also design handling of images, other blobs within the interface.
Design for expansion...  design a versioning system for your protocol....
(compare to IP:    IPv1 ..v3 were test networks, IPv4 is quite well known and stood there for 45 years or so.  v5 continued sideways as MPLS, v6 is oncoming.
(they can all travel on the same network if needed, although most routers probably can handle IPv4 & IP v6).

A design most probably needs more work and more detailed specifications.
Scott FellDeveloper & EE Moderator
Fellow 2018
Most Valuable Expert 2013

Commented:
Depending on the size of the file, you could just have server one make a call to server two over https.  Server two will output a json file.  Server one will read the json and store it to disk. No need for ftp.  

You will want to include some type of authentication on server two so it knows for sure it is server one asking for data and not something else. One way I do this is to send a token which is nothing more than a hash of multiple items. Could be a concatenation of a passcode, year, month, day.  Server one passes the sha256 (or higher) to server two.  Server two checks to see if the submitted hash is what it is supposed to be and if it is, runs the page, if not sends nothing to the page.

You mentioned compliance. Do you have any entities you need to comply with? In other words, are you passing health or financial data that needs to comply with certain laws?
Eduardo FuerteDeveloper and Analyst

Author

Commented:
Thank you for the guidance!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial