• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 855
  • Last Modified:

Import large CSV file into TileMill for use in MapBox

I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded.  When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

MapBox file size limit 20MB notice
Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill?  Eventually, I want to publish the map to the web, and I was planning to do that using MapBox.  I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

1 Solution
the message explicitely tells you that you can change this limit. if you want to do a one-shot importation and the filesize is not huge, and you can change the configuration, this is probably the simplest way

you can also split the file into several files that do not exceed the maximum allowed size. this can be automated pretty easily if required

you can try to onvert to shapefile but you might have similar issues as the format will not be much smaller and i have no idea what would be the limit. most likely the 20m will still apply so if your CSV is much bigger, this will probably be a no-go. better try than ask in this case

if you want to try sqlite, you can use sqlite itself to import your data, and use the sqlite db file to feed to the remote app. beware that the situation will likely be the same as the above.

in your case, i'd probably go with splitting the file in the end, but none of the other options look foolish. i do not think it is really a matter of better or worse but rather picking any solution that performs reasonably good and that you feel comfortable with
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now