How To Parse And Upload A CSV File Using Javascript (PapaParse)

Article


author small image Siva Kishore G
Posted : 01 Sep 2022
Modified : 01 Sep 2022
Expert Developers

Introduction

CSV (Comma Separated Values) or TSV (Tab Separated Values) is the most common data classification template next to xls and json. In this tutorial, we will learn how to parse and upload a csv file into an express application.

For this, we will use a powerful library called PapaParse Microsoft's XLSX / XLS and Google Sheets can readily be converted to CSV. If you don't have Microsoft's office suite, upload the xls / xlsx file into the google drive and open it with google sheets.

Below picture is a demonstration of how to convert a xls file to a csv file.

How to convert xls in google to csv

Setup

You must have an express application running, a sample csv, bootstrap 5, papaparse library (download the papaparse.min.js file) and a million dollars (Just kidding!).

<script href="/path-of-folder/papaparse.min.js"></script>

Papaparse

Before we get into the pool... I mean, the working example, it's better to know a little bit about papabear... I mean papaparse. I'm on fire today!

  • Written in JS.
  • Can offload the work to worker thread.
  • Convert CSV to JSON and vice versa.
  • Ability to pause, resume and abort parser on the run.
  • Extensive documentation.
  • Parse local and remote csv files.
  • JQuery helper.
  • Can work with any delimiter including custom ones.
  • Stream data in case of a large file. (Step option)

Parse CSV File

I like working with bootstrap as I can style my file input element easily. Ok then, let's get into the example.

Since we are only working with CSV files, it's better to restrict the file input tag to .csv using the accept attribute.

<input type="file" accept=".csv" id="csv" class="form-control">

The below function accepts the file supplied from the input element and parses the data step by step while appending the header to each row. The result will be a json with appropriate headers.

var file = document.getElementById("cvs").files[0]
Papa.parse(file, {
  header : true,
	step : function(results,parser) {
    console.log("Result", results.data)
	},
  complete : function(results, file) {
    console.log("Parsing Cmpleted")
  }
});

Upload CSV (one row at a time)

PapaParse has functions like pause, resume and abort which are extremely helpful in this scenario. Let's say, we want to upload the csv into a database. The application will crash if we cram all the data once. So, we will wait for the response from the server for every row in the CSV uploaded. See the example below.

Papa.parse(file, {
  header:true,
	step: function(results,parser) {
    parser.pause()
    $.ajax({
      url: '/<UPLOAD-PATH>',
      type: 'post',
      cache: false,
      data: {row:results.data},
      success: function(data){
        parser.resume()
      },
      error: function(err){
        parser.abort()
      }
    })
	},
  complete: function(results, file) {
    console.log("Parsing Completed")
  }
});

Conclusion

This is an in browser example which is prone to errors if you lost power, or lost internet or simply closed the page. If you worry about this, then you can simply upload the complete file to the server and have the server take care of the uploading part. I suggest offloading the server uploading process to a new node or lambda in case of AWS. The idea here is to keep the server free from computations.



Post a comment

I promise, I keep it clean *

Comments

Alert SVG Cookie Consent

This website uses cookies and similar technologies, to enhance your browsing experience and provide personalized recommendations. By continuing to use our website, you agree to our Privacy policy.