In this short tutorial you will see how you can save binary data, images, in document NoSQL database (CouchDB).
Table of contents:
- Problem description
- Python and images - short introduction
- CouchDB as images store - short inroduction
Problem description
The task is to create an application that generates images consisting of smaller images (mosaics) -- compare, for example, Top Three DON'Ts of Photo Mosaic Design
Requirements
- The images used to generate the resulting image (let's call them thumbnails) should be stored in the CouchDB database (hereinafter referred to as the database).
- There should be at least one view in the database (view, see: Working example, 1.2 : CouchDB basics - querying ) used to search for a matching thumbnail (we do not download all data from the database and do not search them locally).
- The source image can be a rectangle (we do not assume it is a square). li>
- We do not assume in the application any requirements related to the dimensions of the source image (they can be any, of course within reason).
- The application should have a web interface (communication with the application via a web browser).
- Using the web interface you should be able to:
- add one image to the database; li>
- add multiple images in the ZIP archive (optional functionality to grade 5.0); li>
- select the source image that will be used to create the mosaic. li>
Sources
- Working with Images in Python
- Pillow tutorial
- Tutorial. Using the Image class
- Uploading images in Django
- Working with zip files in Python
- Working with PDF files in Python
Python and images - short introduction
- Run Anaconda Navigator with command:
1ubd@ubd-virtual-machine:~$ anaconda-navigator
and installpillow
. Note: remember to do this for a correct environment. - When
pillow
is installed activate your environment (test_dev
in our case)12ubd@ubd-virtual-machine:~$ conda activate test_env(test_env) ubd@ubd-virtual-machine:~$ - Create directory (folder) for our image tests
12(test_env) ubd@ubd-virtual-machine:~$ cd Pulpit/code/(test_env) ubd@ubd-virtual-machine:~/Pulpit/code$ mkdir images
- Download some images. You can use
curl
and download eight imagesfotka01.png
, ...,fotka08.png
fromhttp://fulmanski.pl/zajecia/ssn/cwiczenia/fotki/
12345678(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O http://fulmanski.pl/zajecia/ssn/cwiczenia/fotki/fotka01.png% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed100 95205 100 95205 0 0 989k 0 --:--:-- --:--:-- --:--:-- 989k(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ls -lrazem 100-rw-rw-r-- 1 ubd ubd 95205 mar 18 23:01 fotka01.pngRepeat this for the next eight images.
- Create
img_process_test_01.py
script file located in~/Pulpit/code/images
directory123Create an empty Python script file named <code>img_process_test_01.py(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ touch img_process_test_01.py(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ chmod 755 img_process_test_01.py - Put the following Python code in it
NOTE: The first line in the following source code points to Python we will use and is system/user dependent - it is highly probable that you should use different path according to your settings.:1234567891011121314#!/home/ubd/anaconda3/envs/test_env/bin/python# For various printing styles see# /home/ubd/anaconda3/envs/test_env/bin/pythonfrom PIL import Imagefilename = "fotka04.png"with Image.open(filename) as image:width, height = image.sizeprint("Image width: %s, height:%s" % (width, height))image.show() - Test script in terminal
12(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ./img_process_test_01.pyImage width: 256, height:256
CouchDB as images store - short inroduction
- Create database for our images
12345678910111213(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs["_replicator","_users"](test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://127.0.0.1:5984/images{"error":"unauthorized","reason":"You are not a server admin."}(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://admin:admin@127.0.0.1:5984/images{"ok":true}(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs["_replicator","_users","images"] - Verify in Fauxton -- open web browser and use the following URL to connect with Fauxton
http://127.0.0.1:5984/_utils/
- Add user (user:password:
images_db_user
:pass
)
123456789101112131415161718192021(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://localhost:5984/_users/org.couchdb.user:images_db_user \> -H "Accept: application/json" \> -H "Content-Type: application/json" \> -d '{"name": "images_db_user", "password": "pass", "roles": [], "type": "user"}'{"ok":true,"id":"org.couchdb.user:images_db_user","rev":"1-bea77e75f1f39e08fe49598b18a56f66"}(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://admin:admin@localhost:5984/_users/org.couchdb.user:images_db_user{"_id":"org.couchdb.user:images_db_user","_rev":"1-bea77e75f1f39e08fe49598b18a56f66","name":"images_db_user","roles":[],"type":"user","password_scheme":"pbkdf2","iterations":10,"derived_key":"8d7a5d4df63aabe4fdff5913677993b6f107c49c","salt":"d4941bc472c60738fd4d6fe1bf7882db"} - Verify in Fauxton if this user exists
- Add
images_db_user
to the members ofimages
database. It can be easily done with Fauxton.
Now onlyimages_db_user
user should hav an acess toimages
database
12345678910111213141516171819202122232425262728293031(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://localhost:5984/images{"error":"unauthorized","reason":"You are not authorized to access this db."}(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images{"db_name":"images","purge_seq":"0-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMeC5BkeACk_gNBViIDQbUHIGrvE6N2AUTtfmLUNkDUzsevNikBSCbVE3RrkgNIXTxhdQogdfYE1SUyJMlDFGUBAD9sXo4","update_seq":"8-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8____sxIZ8ShKUgCSSfaE1TmA1MUTVpcAUldPUF0eC5BkaABSQKXziVG7AKJ2PzFqD0DU3idG7QOIWpB7swBjSl6W","sizes":{"file":66784,"external":0,"active":0},"other":{"data_size":0},"doc_del_count":0,"doc_count":0,"disk_size":66784,"disk_format_version":7,"data_size":0,"compact_running":false,"cluster":{"q":8,"n":1,"w":1,"r":1},"instance_start_time":"0"} - Create (add) a document
123456(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT -u images_db_user:pass http://127.0.0.1:5984/images/"test" -d '{"test" : "1"}'{"ok":true,"id":"test","rev":"1-332ff5a5a0c81b2734acfe094ced3c22"}
Verify in Fauxton
- Add an attachment
We want to attach a file namedfotka04.png
, to the previously created document with idtest
, in the database namedimages
. Before that, we have to fetch the data of the document with idtest
to get its current rev id_rev
as shown below.
123456(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test{"_id":"test","_rev":"1-332ff5a5a0c81b2734acfe094ced3c22","test":"1"}
Having current revision id (1-332ff5a5a0c81b2734acfe094ced3c22
) we can upload an image
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -vX PUT http://images_db_user:pass@localhost:5984/images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 --data-binary @fotka04.png -H "ContentType: image/png"* Trying 127.0.0.1...* TCP_NODELAY set* Connected to localhost (127.0.0.1) port 5984 (#0)* Server auth using Basic with user 'images_db_user'> PUT /images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 HTTP/1.1> Host: localhost:5984> Authorization: Basic aW1hZ2VfZGJfdXNlcjpwYXNz> User-Agent: curl/7.63.0> Accept: */*> ContentType: image/png> Content-Length: 81921> Content-Type: application/x-www-form-urlencoded> Expect: 100-continue>< HTTP/1.1 100 Continue< Connection: close< Date: Mon, 18 Mar 2019 22:58:55 GMT< Server: MochiWeb/1.0 (Any of you quaids got a smint?)* We are completely uploaded and fine< HTTP/1.1 201 Created< Cache-Control: must-revalidate< Content-Length: 67< Content-Type: application/json< Date: Mon, 18 Mar 2019 22:58:56 GMT< Location: http://localhost:5984/images/test/fotka04.png< Server: CouchDB/2.3.1 (Erlang OTP/19)< X-Couch-Request-ID: ed4ba9c7b3< X-CouchDB-Body-Time: 0<{"ok":true,"id":"test","rev":"2-1c3211398cacaec8c0a3a80625446efa"}* Closing connection 0(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test{"_id":"test","_rev":"2-1c3211398cacaec8c0a3a80625446efa","test":"1","_attachments":{"fotka04.png":{"content_type":"application/x-www-form-urlencoded","revpos":2,"digest":"md5-OOqdKFGexdqz7Wj6qh4Iog==","length":81921,"stub":true}}}
We can verify this also in Fauxton
Attachment can be easily add also with Fauxton
- Get image from CouchDB
1234(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test/fotka04.png --output fotka04_test.png% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed100 81921 100 81921 0 0 4210k 0 --:--:-- --:--:-- --:--:-- 4210k