How to split Large CSV file into multiple pieces?

Hello everyone! How are you doing? 

In my last article we have about “Latest PHP frameworks in 2014?”. Today I will impart to you one of my ongoing experience.

Recently I came across the task where I need to import large csv records into mysql database. It’s quite tough because your phpmyadmin will not allow you to import large file. That csv contains around 10 million records. You may think don't I use mysql LOADINFILE ? Due to some restriction I avoided and choosed my next option as csv spliting. First thing I tried to open that file, but it is displaying error as “File not loaded completely”.  

The reason behind the above error is excel can only handle 65,536 lines per sheet and it cannot open remaining records. I also tried to import this file directly into PhpMyadmin but the maximum time limit has exceeded. So the next thing what I did is I have increased the PHP "max execution time" but it was taking too much time to process it.

Now you must be thinking that How do i solve this problem?

Here in this blog i will show you how to solve this issue with very simple steps. You can easily solve it simply by splitting the large CSV file into smaller chunks. By doing that script will not take much time process.

You can simply download FXFisherman’s CSV Spliter program and run this program in your windows system. Installation is not required.

You can see below window when you run it.

              Now you may browse your csv file which you want to split, specify how many numbers of lines you want to have per csv and also another way you can simply specify max number of pieces. Now simply click “Split Now!”.

It will begin parting your csv into number of pieces. When this process is completed you should see a screen below.

                

Once process is completed you will find a folder contains chunks of csv file into it, in the same directory. 

Thank God! It Saved My Day. 

I was able to import 3 millions of records at the end of the day.

Do you have any other better or faster way to accomplish this job?

Hope this article will be helpful. Please do not forget to drop your comments below and share with your friends. It will encourage me to write many more such article for you. 

Keep visiting for new posts. If you find my articles are useful just follow my blog and subscribe to newsletter.