jasef
asked on
Recommendations for a language for retrieving information from different datasources (Web based)
Hi Experts,
My background is primarily networking so please keep it simple and broad for now :) I am needing some steering as to the best language(s) and methodologies to utilise for the following scenario (I will most likely oversee development of the project, but could possibly be landed with the programming job too):
We have an inhouse SQL Server database that records our clients and their purchases. Much of it is tailored to property details (our main investment 'product' ATM), but this will be expanding to possibly include shares and superannuation.
This info needs to be made available to clients online (securely of course, so will need to manage logons) AND also provide a facility to report on the current value of their investment by interrogating other websites or sources (some may be SQL, others might be using a querying tool provided by the website itself, obviously all these sources are subject to change, so adaptation or at least easy modification of data source(s) and interrogation will need to be a core component).
I suspect this is best done as a two stage process, firstly developing an application to handle the interrogation and reporting of the different datasources, and secondly (more web based) the customer portal and security etc. but have never attempted this kind of project before.
I have also posted this q to web development, but thought the information retrieval app would perhaps be better answered here. I'm really groping in the dark (but not as much fun as usual) so any candle is appreciated.
Thanks for reading!
My background is primarily networking so please keep it simple and broad for now :) I am needing some steering as to the best language(s) and methodologies to utilise for the following scenario (I will most likely oversee development of the project, but could possibly be landed with the programming job too):
We have an inhouse SQL Server database that records our clients and their purchases. Much of it is tailored to property details (our main investment 'product' ATM), but this will be expanding to possibly include shares and superannuation.
This info needs to be made available to clients online (securely of course, so will need to manage logons) AND also provide a facility to report on the current value of their investment by interrogating other websites or sources (some may be SQL, others might be using a querying tool provided by the website itself, obviously all these sources are subject to change, so adaptation or at least easy modification of data source(s) and interrogation will need to be a core component).
I suspect this is best done as a two stage process, firstly developing an application to handle the interrogation and reporting of the different datasources, and secondly (more web based) the customer portal and security etc. but have never attempted this kind of project before.
I have also posted this q to web development, but thought the information retrieval app would perhaps be better answered here. I'm really groping in the dark (but not as much fun as usual) so any candle is appreciated.
Thanks for reading!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Mayank S
.NET seems to be a good option if you're on SQL Server.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
depends alot here jasef
is your environment primarily windows machines, or do you have linux/bsd/unix machines?
what os is the sql server on?
what os are your web servers?
are you familiar with any languages?
.net is very robust, but i try and steer clear because then your locked into windows...
PHP is also a veyr robust and strong language, and i reccommend it if you dont know anything thus far...reasoning:
PHP runs on windows, bsd, linux, unix, mac, etc....it's not as closed as something .net
It also has sql connections (to MySQL) built in.
If your starting from scratch heres my reccommendation:
Setup a secure enviroment for your webserver (seperate from existing network using router and possibly ipsec)
Host MySQL and apache w/ php on a non windows OS (FreeBSD is my server os of choice, linux or any other BSD/UNIX variant is also acceptable...i don't care what ANYONE tells me windows IIS is NOT AS SECURE OR RELIABLE AS APACHE. PERIOD. i'm not going to get into a flame war over it, but look at the track records...they speak for themselves)
With no experience on FreeBSD, i had it setup, apache, mysql, php, and a firewall setup on a machine in a couple hours...it's very easy
If your currently using MS Sql Server, setup a "View" of all the data your going to need replicated and set it nightly or hourly...depending on how often you need it updated. Better yet, you should have the internet accessable SQL db on a seperate machine....
I suggest you do NOT have your production DB EVER, under ANY case accessable via the internet....reasoning being that if someone does hack the site and corrupt your db, it's nothing to just push the production data right back into the internet DB. Simple as that. We currently have several oracle DB's internal and several completely seperate oracle DB's, containing much of the same into, for the web servers.
You had said you were mainly networking orientated? Look up a reg/orange/green network enviroment (it's called several other things..) i will give a quick rundown (forgive me if you know this)
red - internet. untrusted. Firewalled to other zones. No explicit access to orange/green.
orange - INTERNET server zone. Ability to reach red and internet, and may need access from red > orange. No access to green. Green MAY have access to orange.
Green - Further firewalled from orange and red. access to internet (red) through strict security standards. Some clients may have access to orange (i prefer using ipsec for those needing).
If i mixed anything there up or left things out i am sure someone will fill me in on it.
I know i sound very anti MS products in this post, but my feeling is that if its going to be on the internet it shouldent be run on a windows OS.
Bottom line:
Keep production and internet DB seperate
Keep internet server OFF of your LAN (do not keep it on seperate ip range)
Try not to lock yourself into one OS or vender. MS has a habbit of trying to keep you locked into their products.
Look around. If someone is acomplishing something similar, see how they did it.
If i missed anything or you have anymore questions ask away.
is your environment primarily windows machines, or do you have linux/bsd/unix machines?
what os is the sql server on?
what os are your web servers?
are you familiar with any languages?
.net is very robust, but i try and steer clear because then your locked into windows...
PHP is also a veyr robust and strong language, and i reccommend it if you dont know anything thus far...reasoning:
PHP runs on windows, bsd, linux, unix, mac, etc....it's not as closed as something .net
It also has sql connections (to MySQL) built in.
If your starting from scratch heres my reccommendation:
Setup a secure enviroment for your webserver (seperate from existing network using router and possibly ipsec)
Host MySQL and apache w/ php on a non windows OS (FreeBSD is my server os of choice, linux or any other BSD/UNIX variant is also acceptable...i don't care what ANYONE tells me windows IIS is NOT AS SECURE OR RELIABLE AS APACHE. PERIOD. i'm not going to get into a flame war over it, but look at the track records...they speak for themselves)
With no experience on FreeBSD, i had it setup, apache, mysql, php, and a firewall setup on a machine in a couple hours...it's very easy
If your currently using MS Sql Server, setup a "View" of all the data your going to need replicated and set it nightly or hourly...depending on how often you need it updated. Better yet, you should have the internet accessable SQL db on a seperate machine....
I suggest you do NOT have your production DB EVER, under ANY case accessable via the internet....reasoning being that if someone does hack the site and corrupt your db, it's nothing to just push the production data right back into the internet DB. Simple as that. We currently have several oracle DB's internal and several completely seperate oracle DB's, containing much of the same into, for the web servers.
You had said you were mainly networking orientated? Look up a reg/orange/green network enviroment (it's called several other things..) i will give a quick rundown (forgive me if you know this)
red - internet. untrusted. Firewalled to other zones. No explicit access to orange/green.
orange - INTERNET server zone. Ability to reach red and internet, and may need access from red > orange. No access to green. Green MAY have access to orange.
Green - Further firewalled from orange and red. access to internet (red) through strict security standards. Some clients may have access to orange (i prefer using ipsec for those needing).
If i mixed anything there up or left things out i am sure someone will fill me in on it.
I know i sound very anti MS products in this post, but my feeling is that if its going to be on the internet it shouldent be run on a windows OS.
Bottom line:
Keep production and internet DB seperate
Keep internet server OFF of your LAN (do not keep it on seperate ip range)
Try not to lock yourself into one OS or vender. MS has a habbit of trying to keep you locked into their products.
Look around. If someone is acomplishing something similar, see how they did it.
If i missed anything or you have anymore questions ask away.
ASKER
Thanks for your responses everyone.
.NET seems to be the most common answer here (and the only one on the web dev post), however for the areas I know, fcislers' advice mirrors my own thoughts (hosting the server seperately, using an IPSEC etc), so I'm leaning that way;
We will most likely have the database (at least a copy of it) hosted by an ISP, so the MS security aspect isn't EXTREMELY important - it's their problem (I wouldn't use it as a preferred practice, but I'm not going to get blasted for recommending it if that turns out best). Being locked into MS products is also not an extreme issue (unless it won't work with linux browsers etc. etc. which I hope won't be a problem)... So fcisler, with these things in mind, which
Our environment is windows, but that is irrelevant, the clients will be using god knows what, and I can upload from our Win2K Server MS SQLServer database to mySQL or Postgre if that is a superior solution.
I'm familiar a little with SQL, VB, html & javascript, but not enough to count; So I don't mind picking up a new language from scratch as long as I know it's the best choice.
.NET seems to be the most common answer here (and the only one on the web dev post), however for the areas I know, fcislers' advice mirrors my own thoughts (hosting the server seperately, using an IPSEC etc), so I'm leaning that way;
We will most likely have the database (at least a copy of it) hosted by an ISP, so the MS security aspect isn't EXTREMELY important - it's their problem (I wouldn't use it as a preferred practice, but I'm not going to get blasted for recommending it if that turns out best). Being locked into MS products is also not an extreme issue (unless it won't work with linux browsers etc. etc. which I hope won't be a problem)... So fcisler, with these things in mind, which
Our environment is windows, but that is irrelevant, the clients will be using god knows what, and I can upload from our Win2K Server MS SQLServer database to mySQL or Postgre if that is a superior solution.
I'm familiar a little with SQL, VB, html & javascript, but not enough to count; So I don't mind picking up a new language from scratch as long as I know it's the best choice.
>> the clients will be using god knows what
Then I would recommend Java :-)
But I don't know of any Java drivers for SQL Server.
Then I would recommend Java :-)
But I don't know of any Java drivers for SQL Server.
nooo mayankeagle
java, in theory, is a very good idea
in implementation, it just plain out sucks
soooo slow on windows machines
jasef, if your gonna go .net do it! being locked into a technology refered to your server environment, not clients...provided you don't use anything ie only/ns only then your fine.....net is all (mostly) implemented server side
Look up MySQL....it's advantages are mainly for web based development (quick selects, sorts...slow updates...etc) i may be wrong on this, but read up...
good luck
java, in theory, is a very good idea
in implementation, it just plain out sucks
soooo slow on windows machines
jasef, if your gonna go .net do it! being locked into a technology refered to your server environment, not clients...provided you don't use anything ie only/ns only then your fine.....net is all (mostly) implemented server side
Look up MySQL....it's advantages are mainly for web based development (quick selects, sorts...slow updates...etc) i may be wrong on this, but read up...
good luck
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
>> all i stated was that speed on it SUCKS...it's just plain slow
Maybe you should try working with Eclipse.
>> but not on windows (majority of most workstations)
I have been developing Java applications since years using WSAD and Windows - never faced a problem with that. After development, we usually deploy the application to a Solaris or UNIX server because its more secure than Windoce. Hardly takes a few seconds.
>> So with that, there goes your security
I was not talking about de-compilation. I meant that a Java program runs within an instance of the JVM, and the JVM sees to it that it should not interfere with other processes. Neither do malicious processes affect it. Nor does it affect any outside processes.
>> more robust...than anything else
Because it is architecture-neutral. You change your processor, have upgrades on your system, etc - your C/ C++/ C# programs might stop running - but your Java programs will still run as good as ever.
>> after stating there are no drivers for SQL Server
I am not sure if there are or not. But its not the job of Java to provide drivers for all databases. I don't know whether the JdbcOdbcDriver it provides works for SQL Server or not (it works for Oracle and Access, at least). But generally, third parties provide drivers for their databases in the form of JAR files.
>> but i still think it's seriously lacking.
I don't think it lacks anything. It is portable to any platform -> that way I might say that M$ products are lacking much more than it is (if it is lacking anything).
Just wanted to tell you a litte more about Java, nothing else. Since in this case, it is SQL Server, I have already recommended the use of .NET. But just a moment - *if* the client is on Linux or anything other than Windows, you don't have another option but Java.
Maybe you should try working with Eclipse.
>> but not on windows (majority of most workstations)
I have been developing Java applications since years using WSAD and Windows - never faced a problem with that. After development, we usually deploy the application to a Solaris or UNIX server because its more secure than Windoce. Hardly takes a few seconds.
>> So with that, there goes your security
I was not talking about de-compilation. I meant that a Java program runs within an instance of the JVM, and the JVM sees to it that it should not interfere with other processes. Neither do malicious processes affect it. Nor does it affect any outside processes.
>> more robust...than anything else
Because it is architecture-neutral. You change your processor, have upgrades on your system, etc - your C/ C++/ C# programs might stop running - but your Java programs will still run as good as ever.
>> after stating there are no drivers for SQL Server
I am not sure if there are or not. But its not the job of Java to provide drivers for all databases. I don't know whether the JdbcOdbcDriver it provides works for SQL Server or not (it works for Oracle and Access, at least). But generally, third parties provide drivers for their databases in the form of JAR files.
>> but i still think it's seriously lacking.
I don't think it lacks anything. It is portable to any platform -> that way I might say that M$ products are lacking much more than it is (if it is lacking anything).
Just wanted to tell you a litte more about Java, nothing else. Since in this case, it is SQL Server, I have already recommended the use of .NET. But just a moment - *if* the client is on Linux or anything other than Windows, you don't have another option but Java.
ASKER
Hi everyone, thanks for your responses. I am currently having a good look at PHP, might try .NET if having too much trouble on PHP. Would try Java, but seeing as you guru's aren't too sure about SQL Server integration even though likely (no offence to mayankeagle) I'll give it a miss at this stage.
Intendting to awarding points to fcisler. Any objections?
Intendting to awarding points to fcisler. Any objections?
jasef,
I suggested .NET ages ago.....
Dabas
I suggested .NET ages ago.....
Dabas
I found out that there is a Microsoft SQL Server driver (and even if it was not there, it was not Sun's fault ;-) )
BTW, I'm ok with whoever gets the points - whether there's a split or a single accept.
BTW, I'm ok with whoever gets the points - whether there's a split or a single accept.