How To Import Large CSV File In Database Using Laravel 9

WebSolutionStuff | Sep-15-2022 | Categories : Laravel MySQL

In this article, we will see how to import a large CSV file into the database using laravel 9. Here, we will learn laravel 9 to import a large CSV file into the database using seeder. Many times we have a thousand or millions of records or CSV files and we want to store the records in our database. So, we use the cron or queue jobs for it. But if we want to import the CSV file only one time then we use the database seeder to import the records to the database.

So, let's see import a large CSV file into MySQL laravel 9.

Step 1: Install Laravel 9 App
Step 2: Configure a Database
Step 3: Create Migration and Model
Step 4: Make Seeder
Step 5: Update Code in Seeder
Step 6: Run Seeder and Test

 

Step 1: Install Laravel 9 App

In this step, we will create a laravel 9 application using the following command.

composer create-project --prefer-dist laravel/laravel laravel-9-app

 

 

Step 2: Configure a Database

Now, we will configure a database. So, open the .env file.

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=laravel_9_example
DB_USERNAME=root
DB_PASSWORD=root

 

Step 3: Create Migration and Model

In this step, we will create migration and model using the following command.

php artisan make:model Product -m

Add the below code in database/migrations/create_products_table.php:

public function up()
{
    Schema::create('products', function (Blueprint $table) {
        $table->bigIncrements('id');
        $table->string('name');
        $table->string('sku');
        $table->string('price');
        $table->timestamps();
    });
}

Now, run the migration using the below command.

php artisan migrate

 Changes in the app/Models/Product.php file.

<?php
 
namespace App\Models;
 
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
 
class Product extends Model
{
    use HasFactory;
    protected $guarded = [];
}

 

 

Step 4: Make Seeder

In this step, we will create a seeder file to upload a large CSV file to the database.

php artisan make:seeder ProductSeeder

 

Step 5: Update Code in Seeder

Now, update the code to import large CSV fine in laravel or PHP application. So, Open your database\seeders\ProductSeeder.php file.

<?php

  namespace Database\Seeders;

  use Illuminate\Database\Seeder;
  use App\Models\LocalCommunity;
  use Illuminate\Support\Facades\DB;
  use Illuminate\Support\LazyCollection;

  class CommunitySeeder extends Seeder 
  {
  /**
  * Run the database seeds.
  *
  * @return void
  */
  public function run()
  {
    DB::disableQueryLog();
    DB::table('products')->truncate();

    LazyCollection::make(function () {
      $handle = fopen(public_path("products.csv"), 'r');
      
      while (($line = fgetcsv($handle, 4096)) !== false) {
        $dataString = implode(", ", $line);
        $row = explode(';', $dataString);
        yield $row;
      }

      fclose($handle);
    })
    ->skip(1)
    ->chunk(1000)
    ->each(function (LazyCollection $chunk) {
      $records = $chunk->map(function ($row) {
      return [
        "name" => $row[0],
        "sku" => $row[1],
        "price" => $row[2]
      ];
      })->toArray();
      
      DB::table('products')->insert($records);
    });
  }
}

 

 

Step 6: Run Seeder and Test

Now, run your application using the following command.

php artisan serve

Next, run the seeder using the below command;

php artisan db:seed --class=ProductSeeder

 


You might also like :

Recommended Post
Featured Post





Follow us
facebooklogo github instagram twitter