Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 835
  • Last Modified:

Import large CSV file into TileMill for use in MapBox

I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded.  When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

MapBox file size limit 20MB notice
Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill?  Eventually, I want to publish the map to the web, and I was planning to do that using MapBox.  I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

Thanks.
0
DanielAttard
Asked:
DanielAttard
1 Solution
 
skullnobrainsCommented:
the message explicitely tells you that you can change this limit. if you want to do a one-shot importation and the filesize is not huge, and you can change the configuration, this is probably the simplest way

you can also split the file into several files that do not exceed the maximum allowed size. this can be automated pretty easily if required

you can try to onvert to shapefile but you might have similar issues as the format will not be much smaller and i have no idea what would be the limit. most likely the 20m will still apply so if your CSV is much bigger, this will probably be a no-go. better try than ask in this case

if you want to try sqlite, you can use sqlite itself to import your data, and use the sqlite db file to feed to the remote app. beware that the situation will likely be the same as the above.

in your case, i'd probably go with splitting the file in the end, but none of the other options look foolish. i do not think it is really a matter of better or worse but rather picking any solution that performs reasonably good and that you feel comfortable with
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now