提供了编程的基础技术教程

zl程序教程

您现在的位置是：首页 >  工具

当前栏目

[Docker] Storing Container Data In Docker Volumes

Docker in Data container
2023-09-14 08:59:11 时间

Storing data within a container image is one option for automating a container with data, but it requires a copy of the data to be in each container you run. For static files, this can be a waste of resources. Each file might not amount to more than a few megabytes, but once the containers are scaled up to handle a production load, that few megabytes can turn into gigabytes of waste. Instead, you can store one copy of the static files in a Docker volume for easy sharing between containers. In this lab, you will learn how Docker volumes interact with containers. You will do this by creating new volumes and attaching them to containers. You'll then clean up space left by anonymous volumes created automatically by the containers. Finally, you'll learn about backup strategies for your volumes.

 

 

### Check the setup

To start with, we have already fews images downloaded from DockerHub, and we currently don't have any volume

[cloud_user@ip-10-0-1-129 ~]$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
httpd        2.4       6e794a483258   7 days ago    145MB
bash         latest    2552d69c6348   5 weeks ago   12.9MB
postgres     12.1      cf879a45faaa   2 years ago   394MB
[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME

Anonymous volume

Run the postgres image:

docker run --name db1 -d postgres:12.1
docker run --name db1 -d postgres:12.1

Then let's check volume:

[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME
local     45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d
local     affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199

As we can see that, when postgres start, it needs some space to store the data, so it creates anymonus volume to do that.

Now, let's take a close look to the volume:

  • we want to inspect db1's volume
  • since the inspect response is large, we only interest in Mounts information: -f '{{ json .Mounts }}'
  • then we use python -m json.tool to pretty the json
[cloud_user@ip-10-0-1-129 ~]$ docker inspect db1 -f '{{ json .Mounts }}' | python -m json.tool
[
    {
        "Destination": "/var/lib/postgresql/data",
        "Driver": "local",
        "Mode": "",
        "Name": "affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199",
        "Propagation": "",
        "RW": true,
        "Source": "/var/lib/docker/volumes/affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199/_data",
        "Type": "volume"
    }
]

As you can see, the Name: affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199 is the same as our volumes ouput.

--rm flag, clear the container after stop the container

Now let's run a third container with --rm flag

[cloud_user@ip-10-0-1-129 ~]$ docker run --name dbTmp --rm -d postgres:12.1
1956babce36e413f1402400cbe3201f3a88c5748c82cfa18654dccc1efffe6c9
[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME
local     6f8c3d724bcd6c81738d6e2f64bfa65edcfc84170a10ae76ddd9c9ba7015b905
local     45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d
local     affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199

Let's stop the containers

[cloud_user@ip-10-0-1-129 ~]$ docker stop db2 dbTmp
db2
dbTmp
[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME
local     45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d
local     affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199

As you can see, after stop db2, dbTmp, we still have two volume, one for db1, another is for db2.

The volume for dbTmp has been removed with container by --rm flag when container stopped.

Create Docker Volume

Let's create a docker volume to put our website code:

docker volume create webstie

[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME
local     45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d
local     affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199
local     website

Let's copy our website code from /home/cloud_user/widget-factory-inc/web/* to volume /var/lib/docker/volumes/website/_data/

sudo cp -r /home/cloud_user/widget-factory-inc/web/* /var/lib/docker/volumes/website/_data/

verify the files is there:

[cloud_user@ip-10-0-1-129 ~]$ sudo ls -l /var/lib/docker/volumes/website/_data/
total 16
drwxr-xr-x. 2 root root   76 Jan 25 14:23 img
-rw-r--r--. 1 root root 3059 Jan 25 14:23 index.html
-rw-r--r--. 1 root root 2910 Jan 25 14:23 quote.html
-rw-r--r--. 1 root root 2611 Jan 25 14:23 support.html
-rw-r--r--. 1 root root 2645 Jan 25 14:23 widgets.html

Serve the website using httpd

To server the website using httpd and mount website volume to it

docker run --name web1 -d -p 80:80 -v website:/usr/local/apache2/htdocs:ro httpd:2.4

We should be able to see the site.

Now, run with the --rm flag, see what's happen:

docker run --name webTmp -d --rm -v website:/usr/local/apache2/htdocs:ro httpd:2.4

[cloud_user@ip-10-0-1-129 ~]$ docker ps
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                               NAMES
34ac79c977f2   httpd:2.4       "httpd-foreground"       2 seconds ago    Up 1 second     80/tcp                              webTmp
f2e2aa007625   httpd:2.4       "httpd-foreground"       2 minutes ago    Up 2 minutes    0.0.0.0:80->80/tcp, :::80->80/tcp   web1
f08726cd61cc   postgres:12.1   "docker-entrypoint.s…"   23 minutes ago   Up 23 minutes   5432/tcp                            db1

Let's stop the webTmp

[cloud_user@ip-10-0-1-129 ~]$ docker stop webTmp
webTmp
[cloud_user@ip-10-0-1-129 ~]$ docker ps
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                               NAMES
f2e2aa007625   httpd:2.4       "httpd-foreground"       5 minutes ago    Up 5 minutes    0.0.0.0:80->80/tcp, :::80->80/tcp   web1
f08726cd61cc   postgres:12.1   "docker-entrypoint.s…"   26 minutes ago   Up 26 minutes   5432/tcp                            db1
[cloud_user@ip-10-0-1-129 ~]$ docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS                      PORTS                               NAMES
f2e2aa007625   httpd:2.4       "httpd-foreground"       5 minutes ago    Up 5 minutes                0.0.0.0:80->80/tcp, :::80->80/tcp   web1
c00a4b92b0c8   postgres:12.1   "docker-entrypoint.s…"   25 minutes ago   Exited (0) 13 minutes ago                                       db2
f08726cd61cc   postgres:12.1   "docker-entrypoint.s…"   26 minutes ago   Up 26 minutes               5432/tcp                            db1

As we can see, webTmp has gone.

Check the volumes:

[cloud_user@ip-10-0-1-129 ~]$ docker volume ls
DRIVER    VOLUME NAME
local     45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d
local     affa07409189939e60eb3d520092fac428ac9cde54a63d5b4311b4f3694d9199
local     website

Then we want to prune the volumes:

[cloud_user@ip-10-0-1-129 ~]$ docker volume prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B

Actually no volume has been removed! It is because even we stop db2, but the container is still there, so docker won't delete the volume which has link to a container

[cloud_user@ip-10-0-1-129 ~]$ docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS                      PORTS                               NAMES
f2e2aa007625   httpd:2.4       "httpd-foreground"       8 minutes ago    Up 8 minutes                0.0.0.0:80->80/tcp, :::80->80/tcp   web1
c00a4b92b0c8   postgres:12.1   "docker-entrypoint.s…"   29 minutes ago   Exited (0) 17 minutes ago                                       db2
f08726cd61cc   postgres:12.1   "docker-entrypoint.s…"   30 minutes ago   Up 30 minutes               5432/tcp                            db1

Let's remove db2 container

[cloud_user@ip-10-0-1-129 ~]$ docker rm db2
db2
[cloud_user@ip-10-0-1-129 ~]$ docker ps -a
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                               NAMES
f2e2aa007625   httpd:2.4       "httpd-foreground"       10 minutes ago   Up 10 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp   web1
f08726cd61cc   postgres:12.1   "docker-entrypoint.s…"   31 minutes ago   Up 31 minutes   5432/tcp                            db1

Now db2 container has been deleted.

Let's remove the volme again:

[cloud_user@ip-10-0-1-129 ~]$ docker volume prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
45481ab743b77a1ce8bd3f480b50172e5f4d1c31ae51b565a0f371b11d14100d

Total reclaimed space: 41.43MB

Deletion success!

Backup and restore data to a docker volume

Modify volume need sudo

[cloud_user@ip-10-0-1-129 ~]$ sudo su -
[sudo] password for cloud_user:
[root@ip-10-0-1-129 ~]#

First, let's where docker volume is:

[root@ip-10-0-1-129 ~]# docker volume inspect website
[
    {
        "CreatedAt": "2023-01-25T14:23:53-05:00",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/website/_data",
        "Name": "website",
        "Options": {},
        "Scope": "local"
    }
]

Create a zip file of the voluem data

  • tar czf /tmp/website_$(date +%Y-%m-%d-%H%M).tgz create a zip under /tmp folder with date in name
  • -C /var/lib/docker/volumes/website/_data
  • -C /var/lib/docker/volumes/website/_data .: want to traget directory /var/lib/docker/volumes/website/_data and . we want to zip everythings in this folder

tar czf /tmp/website_$(date +%Y-%m-%d-%H%M).tgz -C /var/lib/docker/volumes/website/_data .

Let's see what have been generated:

[root@ip-10-0-1-129 ~]# ls -l /tmp/website_*.tgz
-rw-r--r--. 1 root root 11363 Jan 25 14:47 /tmp/website_2023-01-25-1447.tgz

Let's also see the files:

[root@ip-10-0-1-129 ~]# ls -l /tmp/website_*.tgz
-rw-r--r--. 1 root root 11363 Jan 25 14:47 /tmp/website_2023-01-25-1447.tgz
[root@ip-10-0-1-129 ~]# tar tf /tmp/website_2023-01-25-1447.tgz
./
./img/
./img/LargeWidget.png
./img/MediumWidget.png
./img/SmallWidget.png
./index.html
./quote.html
./support.html
./widgets.html

We have everthing backup! But one thing is not ideal is that, we don't want everytime to use root user to backup data.

exit

Create a backup with non root access

  • run a container with volumen website to container /website folder
  • create a volume with /tmp (where zip files live) to container /backup
  • zip everything under container's /website folder to /backup folder
docker run -it --rm -v website:/website -v /tmp:/backup bash tar czf /backup/website_$(date +%Y-%m-%d-%H-%M).tgz -C /website .

We should be able to see the zip files:

[cloud_user@ip-10-0-1-86 ~]$ ls -l /tmp
total 1212
-rw-r--r--. 1 root root 1212535 Jan 18  2022 aws-cfn-bootstrap-latest.amzn1.noarch.rpm
drwx------. 3 root root      17 Jan 26 02:30 systemd-private-65fe0c5b569d4f54bfac42ae4f2269c1-chronyd.service-xzrmML
-rw-r--r--. 1 root root   11364 Jan 26 02:52 website_2023-01-26-0252.tgz
-rw-r--r--. 1 root root   11351 Jan 26 02:55 website_2023-01-26-0255.tgz

Restore the files

To test our backup we need to use root access again: sudo su -

Go to the docker volume folder: cd /var/lib/docker/volumes/

Then cd into website volume folder: cd website/_data/

We want to remove everything: rm -rf *

Make sure everything has been deleted:

[root@ip-10-0-1-86 _data]# ls -l
total 0

Now we want to unzip the zip file /tmp/website_2023-01-26-0255.tgz into the current folder to restore the data

[root@ip-10-0-1-86 _data]# tar xf /tmp/website_2023-01-26-0255.tgz .
[root@ip-10-0-1-86 _data]# ls -l
total 16
drwxr-xr-x. 2 root root   76 Jan 26 02:46 img
-rw-r--r--. 1 root root 3059 Jan 26 02:46 index.html
-rw-r--r--. 1 root root 2910 Jan 26 02:46 quote.html
-rw-r--r--. 1 root root 2611 Jan 26 02:46 support.html
-rw-r--r--. 1 root root 2645 Jan 26 02:46 widgets.html

OK! now we got our data back

相关文章