Solved

Import large CSV file into TileMill for use in MapBox

Posted on 2013-06-08
1
789 Views
Last Modified: 2013-06-14
I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded.  When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

MapBox file size limit 20MB notice
Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill?  Eventually, I want to publish the map to the web, and I was planning to do that using MapBox.  I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

Thanks.
0
Comment
Question by:DanielAttard
1 Comment
 
LVL 26

Accepted Solution

by:
skullnobrains earned 500 total points
ID: 39231974
the message explicitely tells you that you can change this limit. if you want to do a one-shot importation and the filesize is not huge, and you can change the configuration, this is probably the simplest way

you can also split the file into several files that do not exceed the maximum allowed size. this can be automated pretty easily if required

you can try to onvert to shapefile but you might have similar issues as the format will not be much smaller and i have no idea what would be the limit. most likely the 20m will still apply so if your CSV is much bigger, this will probably be a no-go. better try than ask in this case

if you want to try sqlite, you can use sqlite itself to import your data, and use the sqlite db file to feed to the remote app. beware that the situation will likely be the same as the above.

in your case, i'd probably go with splitting the file in the end, but none of the other options look foolish. i do not think it is really a matter of better or worse but rather picking any solution that performs reasonably good and that you feel comfortable with
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I feel like more and more people want to know how to programmatically convert addresses into geospatial locations. So in this article, I will show you how you can do it with Bing Maps. I'm going to use PowerShell, which is a nice scripting language,…
Introduction This article is designed to assist GIS (Geographic Information System) and GPS (Global Positioning System) developers using ESRI ArcGIS and other spatial information management systems.   For the uninitiated the concept of projectio…
This tutorial demonstrates how to identify and create boundary or building outlines in Google Maps. In this example, I outline the boundaries of an enclosed skatepark within a community park.  Login to your Google Account, then  Google for "Google M…
This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now