Picture in picture

In this short tutorial you will see how you can save binary data, images, in document NoSQL database (CouchDB).

Table of contents:

• Problem description
  • Sources
• Python and images - short introduction
• CouchDB as images store - short inroduction

The task is to create an application that generates images consisting of smaller images (mosaics) -- compare, for example, Top Three DON'Ts of Photo Mosaic Design

Requirements

1. The images used to generate the resulting image (let's call them thumbnails) should be stored in the CouchDB database (hereinafter referred to as the database).
2. There should be at least one view in the database (view, see: Working example, 1.2 : CouchDB basics - querying ) used to search for a matching thumbnail (we do not download all data from the database and do not search them locally).
3. The source image can be a rectangle (we do not assume it is a square).
4. We do not assume in the application any requirements related to the dimensions of the source image (they can be any, of course within reason).
5. The application should have a web interface (communication with the application via a web browser).
6. Using the web interface you should be able to:
   1. add one image to the database;
   2. add multiple images in the ZIP archive (optional functionality to grade 5.0);
   3. select the source image that will be used to create the mosaic.

• Working with Images in Python
• Pillow tutorial
• Tutorial. Using the Image class
• Uploading images in Django
• Working with zip files in Python
• Working with PDF files in Python

1. Run Anaconda Navigator with command:
   
   ubd@ubd-virtual-machine:~$ anaconda-navigator

   1	ubd@ubd-virtual-machine:~$ anaconda-navigator

   
   and install pillow. Note: remember to do this for a correct environment.
2. When pillow is installed activate your environment (test_dev in our case)
   ubd@ubd-virtual-machine:~$ conda activate test_env (test_env) ubd@ubd-virtual-machine:~$

   1 2	ubd@ubd-virtual-machine:~$ conda activate test_env (test_env) ubd@ubd-virtual-machine:~$

3. Create directory (folder) for our image tests
   (test_env) ubd@ubd-virtual-machine:~$ cd Pulpit/code/ (test_env) ubd@ubd-virtual-machine:~/Pulpit/code$ mkdir images

   1 2	(test_env) ubd@ubd-virtual-machine:~$ cd Pulpit/code/ (test_env) ubd@ubd-virtual-machine:~/Pulpit/code$ mkdir images

4. Download some images. You can use curl and download eight images fotka01.png, ..., fotka08.png from http://fulmanski.pl/zajecia/ssn/cwiczenia/fotki/
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O http://fulmanski.pl/zajecia/ssn/cwiczenia/fotki/fotka01.png % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 95205 100 95205 0 0 989k 0 --:--:-- --:--:-- --:--:-- 989k (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ls -l razem 100 -rw-rw-r-- 1 ubd ubd 95205 mar 18 23:01 fotka01.png

   1 2 3 4 5 6 7 8

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -O http://fulmanski.pl/zajecia/ssn/cwiczenia/fotki/fotka01.png   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100 95205  100 95205    0     0   989k      0 --:--:-- --:--:-- --:--:--  989k   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ls -l razem 100 -rw-rw-r-- 1 ubd ubd 95205 mar 18 23:01 fotka01.png

   Repeat this for the next eight images.

5. Create img_process_test_01.py script file located in ~/Pulpit/code/images directory
   Create an empty Python script file named <code>img_process_test_01.py (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ touch img_process_test_01.py (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ chmod 755 img_process_test_01.py

   1 2 3

   Create an empty Python script file named <code>img_process_test_01.py (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ touch img_process_test_01.py (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ chmod 755 img_process_test_01.py

6. Put the following Python code in it
   NOTE: The first line in the following source code points to Python we will use and is system/user dependent - it is highly probable that you should use different path according to your settings.:
   Python

   #!/home/ubd/anaconda3/envs/test_env/bin/python # For various printing styles see # /home/ubd/anaconda3/envs/test_env/bin/python from PIL import Image filename = "fotka04.png" with Image.open(filename) as image: width, height = image.size print("Image width: %s, height:%s" % (width, height)) image.show()

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   #!/home/ubd/anaconda3/envs/test_env/bin/python   # For various printing styles see # /home/ubd/anaconda3/envs/test_env/bin/python   from PIL import Image   filename = "fotka04.png" with Image.open(filename) as image: width, height = image.size print("Image width: %s, height:%s" % (width, height)) image.show()

7. Test script in terminal
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ./img_process_test_01.py Image width: 256, height:256

   1 2	(test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ ./img_process_test_01.py Image width: 256, height:256

1. Create database for our images
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs ["_replicator","_users"] (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://127.0.0.1:5984/images { "error":"unauthorized", "reason":"You are not a server admin." } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://admin:admin@127.0.0.1:5984/images { "ok":true } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs ["_replicator","_users","images"]

   1 2 3 4 5 6 7 8 9 10 11 12 13

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs ["_replicator","_users"] (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://127.0.0.1:5984/images { "error":"unauthorized", "reason":"You are not a server admin." } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://admin:admin@127.0.0.1:5984/images { "ok":true } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://127.0.0.1:5984/_all_dbs ["_replicator","_users","images"]

2. Verify in Fauxton -- open web browser and use the following URL to connect with Fauxton http://127.0.0.1:5984/_utils/
   
3. Add user (user:password: images_db_user:pass)
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://localhost:5984/_users/org.couchdb.user:images_db_user \ > -H "Accept: application/json" \ > -H "Content-Type: application/json" \ > -d '{"name": "images_db_user", "password": "pass", "roles": [], "type": "user"}' { "ok":true, "id":"org.couchdb.user:images_db_user", "rev":"1-bea77e75f1f39e08fe49598b18a56f66" } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://admin:admin@localhost:5984/_users/org.couchdb.user:images_db_user { "_id":"org.couchdb.user:images_db_user", "_rev":"1-bea77e75f1f39e08fe49598b18a56f66", "name":"images_db_user", "roles":[], "type":"user", "password_scheme":"pbkdf2", "iterations":10, "derived_key":"8d7a5d4df63aabe4fdff5913677993b6f107c49c", "salt":"d4941bc472c60738fd4d6fe1bf7882db" }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT http://localhost:5984/_users/org.couchdb.user:images_db_user \ >      -H "Accept: application/json" \ >      -H "Content-Type: application/json" \ >      -d '{"name": "images_db_user", "password": "pass", "roles": [], "type": "user"}' { "ok":true, "id":"org.couchdb.user:images_db_user", "rev":"1-bea77e75f1f39e08fe49598b18a56f66" } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X GET http://admin:admin@localhost:5984/_users/org.couchdb.user:images_db_user { "_id":"org.couchdb.user:images_db_user", "_rev":"1-bea77e75f1f39e08fe49598b18a56f66", "name":"images_db_user", "roles":[], "type":"user", "password_scheme":"pbkdf2", "iterations":10, "derived_key":"8d7a5d4df63aabe4fdff5913677993b6f107c49c", "salt":"d4941bc472c60738fd4d6fe1bf7882db" }

4. Verify in Fauxton if this user exists
   
5. Add images_db_user to the members of images database. It can be easily done with Fauxton.
   
   
   Now only images_db_user user should hav an acess to images database
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://localhost:5984/images { "error":"unauthorized", "reason":"You are not authorized to access this db." } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images { "db_name":"images","purge_seq":"0-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMeC5BkeACk_gNBViIDQbUHIGrvE6N2AUTtfmLUNkDUzsevNikBSCbVE3RrkgNIXTxhdQogdfYE1SUyJMlDFGUBAD9sXo4", "update_seq":"8-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8____sxIZ8ShKUgCSSfaE1TmA1MUTVpcAUldPUF0eC5BkaABSQKXziVG7AKJ2PzFqD0DU3idG7QOIWpB7swBjSl6W", "sizes":{ "file":66784, "external":0, "active":0 }, "other":{ "data_size":0 }, "doc_del_count":0, "doc_count":0, "disk_size":66784, "disk_format_version":7, "data_size":0, "compact_running":false, "cluster":{ "q":8, "n":1, "w":1, "r":1 }, "instance_start_time":"0" }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://localhost:5984/images { "error":"unauthorized", "reason":"You are not authorized to access this db." } (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images { "db_name":"images","purge_seq":"0-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMeC5BkeACk_gNBViIDQbUHIGrvE6N2AUTtfmLUNkDUzsevNikBSCbVE3RrkgNIXTxhdQogdfYE1SUyJMlDFGUBAD9sXo4", "update_seq":"8-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFMiQ5L8____sxIZ8ShKUgCSSfaE1TmA1MUTVpcAUldPUF0eC5BkaABSQKXziVG7AKJ2PzFqD0DU3idG7QOIWpB7swBjSl6W", "sizes":{ "file":66784, "external":0, "active":0 }, "other":{ "data_size":0 }, "doc_del_count":0, "doc_count":0, "disk_size":66784, "disk_format_version":7, "data_size":0, "compact_running":false, "cluster":{ "q":8, "n":1, "w":1, "r":1 }, "instance_start_time":"0" }

6. Create (add) a document
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT -u images_db_user:pass http://127.0.0.1:5984/images/"test" -d '{"test" : "1"}' { "ok":true, "id":"test", "rev":"1-332ff5a5a0c81b2734acfe094ced3c22" }

   1 2 3 4 5 6

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -X PUT -u images_db_user:pass http://127.0.0.1:5984/images/"test" -d '{"test" : "1"}' { "ok":true, "id":"test", "rev":"1-332ff5a5a0c81b2734acfe094ced3c22" }

   
   Verify in Fauxton
   
7. Add an attachment
   We want to attach a file named fotka04.png, to the previously created document with id test, in the database named images. Before that, we have to fetch the data of the document with id test to get its current rev id _rev as shown below.
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test { "_id":"test", "_rev":"1-332ff5a5a0c81b2734acfe094ced3c22", "test":"1" }

   1 2 3 4 5 6

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test { "_id":"test", "_rev":"1-332ff5a5a0c81b2734acfe094ced3c22", "test":"1" }

   
   Having current revision id (1-332ff5a5a0c81b2734acfe094ced3c22) we can upload an image
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -vX PUT http://images_db_user:pass@localhost:5984/images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 --data-binary @fotka04.png -H "ContentType: image/png" * Trying 127.0.0.1... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 5984 (#0) * Server auth using Basic with user 'images_db_user' > PUT /images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 HTTP/1.1 > Host: localhost:5984 > Authorization: Basic aW1hZ2VfZGJfdXNlcjpwYXNz > User-Agent: curl/7.63.0 > Accept: */* > ContentType: image/png > Content-Length: 81921 > Content-Type: application/x-www-form-urlencoded > Expect: 100-continue > < HTTP/1.1 100 Continue < Connection: close < Date: Mon, 18 Mar 2019 22:58:55 GMT < Server: MochiWeb/1.0 (Any of you quaids got a smint?) * We are completely uploaded and fine < HTTP/1.1 201 Created < Cache-Control: must-revalidate < Content-Length: 67 < Content-Type: application/json < Date: Mon, 18 Mar 2019 22:58:56 GMT < Location: http://localhost:5984/images/test/fotka04.png < Server: CouchDB/2.3.1 (Erlang OTP/19) < X-Couch-Request-ID: ed4ba9c7b3 < X-CouchDB-Body-Time: 0 < { "ok":true, "id":"test", "rev":"2-1c3211398cacaec8c0a3a80625446efa" } * Closing connection 0 (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test { "_id":"test", "_rev":"2-1c3211398cacaec8c0a3a80625446efa", "test":"1", "_attachments":{ "fotka04.png":{ "content_type":"application/x-www-form-urlencoded", "revpos":2, "digest":"md5-OOqdKFGexdqz7Wj6qh4Iog==", "length":81921, "stub":true } } }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl -vX PUT http://images_db_user:pass@localhost:5984/images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 --data-binary @fotka04.png -H "ContentType: image/png" *   Trying 127.0.0.1... * TCP_NODELAY set * Connected to localhost (127.0.0.1) port 5984 (#0) * Server auth using Basic with user 'images_db_user' > PUT /images/test/fotka04.png?rev=1-332ff5a5a0c81b2734acfe094ced3c22 HTTP/1.1 > Host: localhost:5984 > Authorization: Basic aW1hZ2VfZGJfdXNlcjpwYXNz > User-Agent: curl/7.63.0 > Accept: */* > ContentType: image/png > Content-Length: 81921 > Content-Type: application/x-www-form-urlencoded > Expect: 100-continue > < HTTP/1.1 100 Continue < Connection: close < Date: Mon, 18 Mar 2019 22:58:55 GMT < Server: MochiWeb/1.0 (Any of you quaids got a smint?) * We are completely uploaded and fine < HTTP/1.1 201 Created < Cache-Control: must-revalidate < Content-Length: 67 < Content-Type: application/json < Date: Mon, 18 Mar 2019 22:58:56 GMT < Location: http://localhost:5984/images/test/fotka04.png < Server: CouchDB/2.3.1 (Erlang OTP/19) < X-Couch-Request-ID: ed4ba9c7b3 < X-CouchDB-Body-Time: 0 < { "ok":true, "id":"test", "rev":"2-1c3211398cacaec8c0a3a80625446efa" } * Closing connection 0 (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test { "_id":"test", "_rev":"2-1c3211398cacaec8c0a3a80625446efa", "test":"1", "_attachments":{ "fotka04.png":{ "content_type":"application/x-www-form-urlencoded", "revpos":2, "digest":"md5-OOqdKFGexdqz7Wj6qh4Iog==", "length":81921, "stub":true } } }

   
   We can verify this also in Fauxton
   
   
   Attachment can be easily add also with Fauxton
   
8. Get image from CouchDB
   
   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test/fotka04.png --output fotka04_test.png % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 81921 100 81921 0 0 4210k 0 --:--:-- --:--:-- --:--:-- 4210k

   1 2 3 4

   (test_env) ubd@ubd-virtual-machine:~/Pulpit/code/images$ curl http://images_db_user:pass@localhost:5984/images/test/fotka04.png --output fotka04_test.png   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100 81921  100 81921    0     0  4210k      0 --:--:-- --:--:-- --:--:-- 4210k