Page 1 of 1

Question about building a tiny .qcow2 for different uses

Posted: Sun Jul 23, 2017 7:14 pm
by Doctor Venture
As a GNS3 user/contributor, I see a LOT of posts from people that are perplexed about why the changes they make to their docker containers vanish after they shut down their projects and restart it.

From my limited understanding of Docker, only things that were present in the dockerfile used during build-time will be persistent. Sure, there's a read/write part, where folks can install python, etc..., but those changes aren't permanent.

I've built qemu .qcow2 images before, that I had made for specific uses, like one for Ansible, one for ZTP, one for Puppet, a AAA server, and others, but folks have balked at downloading a 1-2GB image file. I even found a small 256MB debian wheezy .qcow2 image someone made, but it'll bloat up to 1-2GB, after all the dependencies are installed. Does anyone know of a linux distro that I could use to create similar server images, but something more in the 500MB? Once I figure out how to make docker containers, I could do something similar, but then folks would squawk about their Ansible playbooks, or other files going AWOL. Think I could do something similar with Arch or Alpine Linux?

Re: Question about building a tiny .qcow2 for different uses

Posted: Sun Jul 23, 2017 9:40 pm
by blahsaysblah
Have you tried Alpine as your base image? Though it seems you need to cut down your dependencies and not the base image...

Re: Question about building a tiny .qcow2 for different uses

Posted: Sun Jul 23, 2017 9:51 pm
by Doctor Venture
blahsaysblah wrote:
Have you tried Alpine as your base image? Though it seems you need to cut down your dependencies and not the base image...



I just gave it a shot a few minutes ago. Ansible seemed to install with "apk add ansible", and the resulting image was only 326MB (I used the alpine-virt .iso image to make the .qcow2 disk image). I need to test it out, though, since I also installed nano, and it's not on there. I had to use vi to set up a static ip, and configure it to use a serial console, instead of needing VNC.

Re: Question about building a tiny .qcow2 for different uses

Posted: Mon Jul 24, 2017 8:01 am
by notfred
You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.

Re: Question about building a tiny .qcow2 for different uses

Posted: Mon Jul 24, 2017 2:54 pm
by Doctor Venture
notfred wrote:
You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.


I haven't done something like that in almost 17 years. I'll definitely need to either find my "Building Linux from the ground up" book, or a more recent edition, since a lot of this is waaaay out of my wheelhouse. I know just enough about Ubuntu, Debian, Fedora, WindRiver, and Montevista to be dangerous, and that's just because the last four are used in some of the network gear I install/provision.

I really wish Alcatel-Lucent/Nokia would make a trial version of VSR freely available for learning purposes. The last version of their TiMOS sim I have is 13.0R.6, and is really limited. I'd love to try even a bandwidth crippled version of VSR, just because I'd love to see what it can do. I've *seen* their 7750 Service Routers, but only from a distance. I wasn't allowed anywhere near them. :-?

Re: Question about building a tiny .qcow2 for different uses

Posted: Mon Jul 24, 2017 3:18 pm
by just brew it!
notfred wrote:
You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.

FWIW the kpartx tool is quite useful for doing stuff like this. It saves you from having to manually specify the partition offsets by automatically creating device entries for the partitions under /dev/mapper.

Re: Question about building a tiny .qcow2 for different uses

Posted: Mon Jul 24, 2017 3:31 pm
by bthylafh
The canonical way to create Linux from scratch is... Linux From Scratch.

http://www.linuxfromscratch.org/lfs/read.html

On a lark I did that about three weeks ago; took a few hours, most of which was waiting for glibc to compile, twice. I wouldn't suggest it for making a Linux you use on a daily basis - having a good dependency-resolving package manager is just too much of a convenience - but it's an interesting thing to try.

Re: Question about building a tiny .qcow2 for different uses

Posted: Mon Jul 24, 2017 3:40 pm
by Doctor Venture
bthylafh wrote:
The canonical way to create Linux from scratch is... Linux From Scratch.

http://www.linuxfromscratch.org/lfs/read.html

On a lark I did that about three weeks ago; took a few hours, most of which was waiting for glibc to compile, twice. I wouldn't suggest it for making a Linux you use on a daily basis - having a good dependency-resolving package manager is just too much of a convenience - but it's an interesting thing to try.


I'll check that out. Thanks! :D

I've been avoiding building ubuntu server VMs, due to the sheer bloat (that's why I'm hoping Alpine-virt pans out), but if Linux from Scratch pans out, that'll be cool, too.

I did try that canonical "automated" Openstack build process, but it always failed on one of the final steps. It was much simpler to just make an Openstack controller, and storage and compute nodes by hand (I was testing out a section of a data center topology I was building).

Re: Question about building a tiny .qcow2 for different uses

Posted: Tue Jul 25, 2017 8:19 am
by notfred
Doing something like Linux From Scratch teaches you a lot about how the system bootstraps up and gets running, very useful. I did something like that with my diskless folding thing that I had years ago. The source is still out there https://sourceforge.net/p/foldingcd/code/HEAD/tree/

And thanks JBI for the pointer to kpartx, that's a lot easier than calculating offsets!

Re: Question about building a tiny .qcow2 for different uses

Posted: Wed Nov 15, 2017 3:07 am
by rona rosen1
Docker containers do persist after exits, unless the
docker run 
command used the
--rm
option. So if you exit the shell, then restart and re-attach to that container the earlier files are still there. However, Docker management best practices are to consider containers as read-only and generate new images with a Dockerfile and the
docker build
command. While you are in a container, you can use the
docker commit 
command, which will save the container contents to a new image, it's just not recommended to use that as a way of "saving" your container.

There are ways to reduce docker image sizes, such as using a smaller base image, minimize layers, using the
no-install-recommends
option when running the
apt-get install
command. Docker has a lot of flexibility, but it's also a new way of thinking about computing so using it like a standard storage device can get you in trouble. Some best practices models for Docker have the data kept externally to the container, like with host volume mounts. There's a cool wiki that has all sorts of information about containers, and it has resources on how to create docker images if you are interested, the Container Technology Wiki

Apologies, I'm not sure why that "Code, Select All" weird formatting is in there. Sorry. I'm a noob.