Flora Edwards
asked on
Where to find a large 2 million rows txt file
Where can i download a large dataset in txt file format to import in Excel for testing. Idealy it should be more than 2 million rows.
You could just create one with a vbscript (or powershell, or possibly even batch) and a loop
This comes to mind. Password list with 1,493,677,782 lines
https://crackstation.net/buy-crackstation-wordlist-password-cracking-dictionary.htm
https://crackstation.net/buy-crackstation-wordlist-password-cracking-dictionary.htm
Hi Flora,
You probably already know this, but if importing to an Excel worksheet, the limit is about 1m rows.
You can split into multiple worksheets, or load into a VBA array, or read it in one line at a time - lots of other options, but if you want to place it into a single worksheet there is that limit.
Alan.
You probably already know this, but if importing to an Excel worksheet, the limit is about 1m rows.
You can split into multiple worksheets, or load into a VBA array, or read it in one line at a time - lots of other options, but if you want to place it into a single worksheet there is that limit.
Alan.
ASKER
Lee,
How can that be done? I have no clue, can you please help?
How can that be done? I have no clue, can you please help?
ASKER
Thanks Alan.
Yes, I know that. but i need that for another purpose. for powerquery
Yes, I know that. but i need that for another purpose. for powerquery
ASKER
Thanks Shaun but that link you shared, takes me to torrent.
i do not have torrent and i am scared of using torrent. is there any other way to donwload it without use of torrent?
i do not have torrent and i am scared of using torrent. is there any other way to donwload it without use of torrent?
There are sites like https://www.generatedata.com/ but only allow 100 at a time. For a small fee you can purchase a million. https://www.briandunning.com/sample-data/.
If it were me, I would create a script that can generate random for random lengths from min to max and repeat 2 million times which is what Lee suggested already.
If it were me, I would create a script that can generate random for random lengths from min to max and repeat 2 million times which is what Lee suggested already.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
i do not have torrent and i am scared of using torrent. is there any other way to donwload it without use of torrent?Really why? It is a method of distributing files, even Linux distros use it. All you need is uTorrent
www.utorrent.com/
Hi Flora,
I agree with Shaun - torrents are just a more efficient way of moving files around than a direct (HTTP) download.
Probably less risky in many respects, if you can see many people acting as seeds, since it is generally unlikely that loads of people will be involved if the file is malicious - something you can never know with a direct download.
For large files it is much more efficient and reliable.
Alan.
I agree with Shaun - torrents are just a more efficient way of moving files around than a direct (HTTP) download.
Probably less risky in many respects, if you can see many people acting as seeds, since it is generally unlikely that loads of people will be involved if the file is malicious - something you can never know with a direct download.
For large files it is much more efficient and reliable.
Alan.
ASKER
Thank you very much Lee, This was really Amazing!
what programming language did you use in your code?
what programming language did you use in your code?
That's just a batch script using a for loop.
ASKER
Thanks. is there any reference or website that teaches this script?
from the command prompt, type
for /?
For example, you can also ping an entire network subnet using the for command
for /l %a in (1,1,254) do @ping -n 1 -w 100 192.168.1.%a | find /i "reply"
In short, the above uses the for command as a number sequence to start with the number 1, increase by 1, and end at 254. Then it runs ("does") a ping command. Finally the output of the ping command is directed to the find command (rather than the screen) using the | character and it looks for all lines with the word "reply" in it (the /i says to ignore case so "Reply" and "reply" is the same).
One of my favorite web sites for batch scripting is http://www.robvanderwoude.com (See Scripting languages at left). I also like www.ss64.com for reference (it's like an enhanced command /? display only with some added comments and examples).
for /?
For example, you can also ping an entire network subnet using the for command
for /l %a in (1,1,254) do @ping -n 1 -w 100 192.168.1.%a | find /i "reply"
In short, the above uses the for command as a number sequence to start with the number 1, increase by 1, and end at 254. Then it runs ("does") a ping command. Finally the output of the ping command is directed to the find command (rather than the screen) using the | character and it looks for all lines with the word "reply" in it (the /i says to ignore case so "Reply" and "reply" is the same).
One of my favorite web sites for batch scripting is http://www.robvanderwoude.com (See Scripting languages at left). I also like www.ss64.com for reference (it's like an enhanced command /? display only with some added comments and examples).
ASKER
Many Thanks Lee!
http://www.sample-videos.com/download-sample-text-file.php