?
Solved

does this project can be done on linux fedora

Posted on 2007-07-24
9
Medium Priority
?
380 Views
Last Modified: 2010-04-20
Hi all,
I need your help as proffessionals in that :
I got thoes requirement from the system analyst
- estimated number of concurrent users per second on the web application is 5 million user
- estimated number of records per database is 40 million record
- record size is 20 kilo byte

I'm responsible for technical specification but I'm newbie
- does  linux fedora 6 can take that huge amount of data and that huge amount of users per second
if so how about the stability and security and performance and speed in that case is it good enough or it 'll be poor???
best Regards
HG
0
Comment
Question by:shang3000
  • 4
  • 2
  • 2
  • +1
9 Comments
 
LVL 27

Expert Comment

by:Nopius
ID: 19556830
> does  linux fedora 6 can take that huge amount of data and that huge amount of users per second

Short answer: NO.

That's impossible to run on a single Linux machine because of:

1) It cant' handle 5 million requests per second (really no one single machine on any OS can handle such amount of requests)

2) I don't know any Free DB with 820Tb database ability. I'm sure it can't be PostgreSQL (with it's limit to 32Tb). Not sure about MySQL (I know it can handle up to 2^64 rows on MyISAM storage engine). Not sure about BerkeleyDB .

3) The only 2 filesystem types in Linux can handle a volume with 820Tb file are JFS and XFS. But on a 64-bit Linux. http://en.wikipedia.org/wiki/Comparison_of_file_systems

4) It's possible to find an (multi-shelf) array with total storage capacity of 850Tb. But I don't sure about maximum LUN size for that array (for example Sun StorageTek: http://sunsolve.sun.com/handbook_pub/Systems/9990V/spec.html
with 4 shelves).

BUT the long answer is YES. Good example of Linux driven implementation with such abilities is a google.com.
BUT they use self-developed non standard clustering solution with a distributed storage. Probably they don't have Fedora Core on their machines, but it doesn't matter which distribution to use when we say about kernel and physical limits (network/disk/cpu load). So it may be possible with Linux, but requires investments compared to google's funds.


0
 
LVL 16

Expert Comment

by:xDamox
ID: 19557997
Hi,

As Nopius states the short answer is no. You could try an simulate 5 million requests using the Apache benchmark utility and see what happends
try issuing the command as root ab
0
 
LVL 16

Expert Comment

by:xDamox
ID: 19558016
O and a little note on that google example, I have heard they use a customized version of RHEL
0
Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

 
LVL 7

Expert Comment

by:ezaton
ID: 19559873
They have a customized version of Redhat 7.x.
But it's nto the issue - the issue is that, as said before, no single machine can support such a scenario. The setup for such an enormous setups is with large-sized load balancers, and large sized servers (talk about multi-cores. Lots of them).
We have tested in the past running SSL endpoint on rather modern hardware running linux, and we were able to reach 3K TPS with SSLv2. Just to give you some perspective.

I would not have used unsupported system, such as Fedora core for such a system. Assuming you will need one of the larger DBs, such as Oracle or Sybase, or DB2, or the likes, you will want a fully supported system. With the relatively low price of RedHat (would be about 1400$ per server) this should be considered a must.

You're talking about one of the larger RACs available, I think. This is a huge monster.
0
 

Author Comment

by:shang3000
ID: 19566546
Hi Nopius,
>>>That's impossible to run on a single Linux machine
can you please tell me what are the maximum limit of a single Linux machine

>>>BUT the long answer is YES. Good example of Linux driven implementation >>>with such abilities is a google.com.
can you guide me or give me a direction to follow to reach that example
just mintion a site, a link, an article, a hosting company that can do this any thing...
0
 
LVL 27

Accepted Solution

by:
Nopius earned 2000 total points
ID: 19571371
> can you please tell me what are the maximum limit of a single Linux machine
65535 concurrent connections (even not connections per second) is the maximum for Linux with firewall enabled.

> can you guide me or give me a direction to follow to reach that example

Google Inc.
1600 Amphitheatre Parkway
Mountain View CA 94043
tel.: (650) 253-0000
fax: (650) 253-0001

or (from RIPE database)
OrgTechName:   Google Inc.
OrgTechPhone:  +1-650-318-0200
OrgTechEmail:  arin-contact at google . com

You may ask them how did they reach such limits, but believe me it's a proprietary closed information. It seems they dont' use any hosting provider, just have direct high speed links to different internet providers and they host their server farm on their own territory.

Another approach - find a good IT integration company who specialized on extra large network clusters and ask them.

You have zero chances to find hosting company that satisfy your requirements (at least I don't know any, but we may wait for other expert's answers). So you should build your own cluster oriented hosting solution.

0
 

Author Comment

by:shang3000
ID: 19575156
hi Nopius,
thanks again for your advice
---> can you please tell me what are the maximum limit of a single Linux machine
>>>65535 concurrent connections (even not connections per second) is the
>>>maximum for Linux with firewall enabled.
can you please tell me the number of concurrent connection per second the single linux machine can take?
0
 
LVL 27

Expert Comment

by:Nopius
ID: 19579439
> can you please tell me the number of concurrent connection per second the single linux machine can take?

I can't, that much depends on your hardware and application that handles connection, you should test it in your environment.
0
 
LVL 27

Expert Comment

by:Nopius
ID: 19579675
As a clue, maximum connection rate (if you have no limitations) may vary from 30 to 3000 connections/sec.

In default Linux kernel config look into /proc/sys/net/ipv4/tcp_max_syn_backlog - that's a number of queued 'SYN' requests, that are still unacknowledged (http://www.uwsg.iu.edu/hypermail/linux/net/0111.3/0009.html).

By default I have 1024, some heavy loaded systems admins setup it to several thousands, say 8096 (http://bugs.centos.org/view.php?id=1577), it may increase the maximum number of connections per second, but all these connections will be unmanaged if your application is not fast enough.

For example for Apache WEB server 1000 requests/second on 1000 concurrent connections is very good result (but typically this value is lower).

0

Featured Post

Get quick recovery of individual SharePoint items

Free tool – Veeam Explorer for Microsoft SharePoint, enables fast, easy restores of SharePoint sites, documents, libraries and lists — all with no agents to manage and no additional licenses to buy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension (http://www.experts-exchange.com/discussions/210281/Attachments-with-no-extension.html). This reminded me of questions tha…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses
Course of the Month13 days, 9 hours left to enroll

750 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question