Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


Import large CSV file into TileMill for use in MapBox

Posted on 2013-06-08
Medium Priority
Last Modified: 2013-06-14
I am beginning to experiment with using MapBox and TileMill, and what I would like to do is map 400,000 addresses in a CSV file which have been pre-geocoded.  When I try to add this 100mb CSV file as a layer into MapBox, I receive an error telling me that the CSV file is greater than 20mb and apparently this is a problem.

MapBox file size limit 20MB notice
Can someone point me in the right direction in terms of what is the best way to get these 400k records into TileMill?  Eventually, I want to publish the map to the web, and I was planning to do that using MapBox.  I saw a program for converting CSV to a shapefile, and wondering whether this is the best approach.

Question by:DanielAttard
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
1 Comment
LVL 27

Accepted Solution

skullnobrains earned 2000 total points
ID: 39231974
the message explicitely tells you that you can change this limit. if you want to do a one-shot importation and the filesize is not huge, and you can change the configuration, this is probably the simplest way

you can also split the file into several files that do not exceed the maximum allowed size. this can be automated pretty easily if required

you can try to onvert to shapefile but you might have similar issues as the format will not be much smaller and i have no idea what would be the limit. most likely the 20m will still apply so if your CSV is much bigger, this will probably be a no-go. better try than ask in this case

if you want to try sqlite, you can use sqlite itself to import your data, and use the sqlite db file to feed to the remote app. beware that the situation will likely be the same as the above.

in your case, i'd probably go with splitting the file in the end, but none of the other options look foolish. i do not think it is really a matter of better or worse but rather picking any solution that performs reasonably good and that you feel comfortable with

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I feel like more and more people want to know how to programmatically convert addresses into geospatial locations. So in this article, I will show you how you can do it with Bing Maps. I'm going to use PowerShell, which is a nice scripting language,…
Introduction This article is designed to assist GIS (Geographic Information System) and GPS (Global Positioning System) developers using ESRI ArcGIS and other spatial information management systems.   For the uninitiated the concept of projectio…
This tutorial demonstrates how to identify and create boundary or building outlines in Google Maps. In this example, I outline the boundaries of an enclosed skatepark within a community park.  Login to your Google Account, then  Google for "Google M…
This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question