Automate data import process
Posted on 2014-09-07
I'm using the Salesforce platform and I currently have multiple flat files that need to be imported daily. Currently, I manually import the files using data loader. I am doing very simple imports to Accounts and Opportunities objects in Salesforce.
The process is two prong. First, we have a process that exports the data from another database to a flat file (csv) and copies it over to a FTP site. Then I will grab the flat file (csv) from the FTP site and load into Salesforce.
I am looking at ways to automate this process with no user intervention. I am looking at the automated pull approach or an http request / response. Is there any Salesforce examples of how each of these processes would be implemented? I'm sure this is probably a pretty common problem. I don't want to do any command line scripting process and using a middleware solutions like Informatica Cloud isn't on the table at the present time.
I haven't had a ton of integration experience, but as I understand it I have the following choices if I dont want to license a tool, such as Informatica Cloud. I could a) write an apex rest service that would accept a POST with the new data, or b) write an inbound email handler that accepted csv attachments or c) have a scheduled apex job that made an outbound rest/soap call to a rest/soap service on your source system. I am not very familiar with REST, but option "a" seems like a clean solution, but an external web service would have to be written to POST to an APEX rest service that can accept the POST, correct? Option "b" seems like it wouldn't be very viable based on potential size issue of the flat files and parsing the csv file. Option "c" seems reasonable too, but again, there would need to be an external REST/SOAP rest service that I can make a call to that would need to be built.
Being completely new to integration development (REST), does anyone have any examples they can point me to that demonstrate how to accomplish options "a" and "b"?
Has anyone replaced the dataloader manual loading process in Salesforce with an automated process using REST or SOAP? I'm looking for any guidance on how to accomplish something like this.
Any examples of how to implement this functionality would be immensely helpful.
Thanks in advance for any help.