How to make awesome MongoDB database of barcodes

We’ll choose “Universe-HTT barcode reference” — The largest and most accurate open reference book of bar codes on the Internet!

Imagine you have ubuntu-based instance from DigitalOcean or any other cloud provider. And you already have MongoDB instance on the same machine or inside docker-container.

How import data from release files to database?

Make a folder for our actions:

mkdir Import
cd Import

Get the last release of barcodes:

wget https://github.com/papyrussolution/UhttBarcodeReference/releases/download/20181106/uhtt_barcode_ref_all.7z

Install archiver (If it necessary):

apt-get install p7zip

Unpack the archive in the same directory:

7zr x uhtt_barcode_ref_all.7z

Delete everything excludes the main data-file:

find . ! -name 'uhtt_barcode_ref_all.csv' -type f -exec rm -f {} +

Convert the main csv-file for properly import:

tr "," ";" < uhtt_barcode_ref_all.csv > uhtt_barcode_ref_all_1.csv
tr "\"" "'" < uhtt_barcode_ref_all_1.csv > uhtt_barcode_ref_all_2.csv
tr "\t" "," < uhtt_barcode_ref_all_2.csv > uhtt_barcode_ref_all_final.csv

(we changed the type of dividers and escaped quotes)

Last step it is import to database:

mongoimport -d youbacrodesdb -c barcodes --type csv --file uhtt_barcode_ref_all_final.csv.csv --headerline --host localhost:27017 --authenticationDatabase youauthdb --username 'youname' --password 'youpassword'

After that you’ll get the database with collection “barcodes”, like this:

{
"_id" : ObjectId("5bf1cfadbcf61748215ab5c1"),
"ID" : 1371546,
"UPCEAN" : NumberLong(5060150684272),
"Name" : "Adore Extra sure condoms 12pk",
"CategoryID" : 108335,
"CategoryName" : "Гигиена (folder)/Презервативы",
"BrandID" : 0,
"BrandName" : ""
}

{
"_id" : ObjectId("5bf1cfadbcf61748215ab5c2"),
"ID" : 1991599,
"UPCEAN" : NumberLong(7350022277762),
"Name" : "Adore me pleasure set (net)",
"CategoryID" : 1980172,
"CategoryName" : "Adult/All Novelties",
"BrandID" : 0,
"BrandName" : ""
}

Don’t forget add “UPCEAN” field to indexes!