Solved

Import large CSV file into TileMill for use in MapBox

Posted on 2013-06-08
1
785 Views
Last Modified: 2013-06-14
I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded.  When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

MapBox file size limit 20MB notice
Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill?  Eventually, I want to publish the map to the web, and I was planning to do that using MapBox.  I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

Thanks.
0
Comment
Question by:DanielAttard
1 Comment
 
LVL 26

Accepted Solution

by:
skullnobrains earned 500 total points
ID: 39231974
the message explicitely tells you that you can change this limit. if you want to do a one-shot importation and the filesize is not huge, and you can change the configuration, this is probably the simplest way

you can also split the file into several files that do not exceed the maximum allowed size. this can be automated pretty easily if required

you can try to onvert to shapefile but you might have similar issues as the format will not be much smaller and i have no idea what would be the limit. most likely the 20m will still apply so if your CSV is much bigger, this will probably be a no-go. better try than ask in this case

if you want to try sqlite, you can use sqlite itself to import your data, and use the sqlite db file to feed to the remote app. beware that the situation will likely be the same as the above.

in your case, i'd probably go with splitting the file in the end, but none of the other options look foolish. i do not think it is really a matter of better or worse but rather picking any solution that performs reasonably good and that you feel comfortable with
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

I feel like more and more people want to know how to programmatically convert addresses into geospatial locations. So in this article, I will show you how you can do it with Bing Maps. I'm going to use PowerShell, which is a nice scripting language,…
Introduction This article is designed to assist GIS (Geographic Information System) and GPS (Global Positioning System) developers using ESRI ArcGIS and other spatial information management systems.   For the uninitiated the concept of projectio…
This tutorial demonstrates how to identify and create boundary or building outlines in Google Maps. In this example, I outline the boundaries of an enclosed skatepark within a community park.  Login to your Google Account, then  Google for "Google M…
This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now