Personal computing discussed

Moderators: renee, SecretSquirrel, notfred

 
Doctor Venture
Gerbil First Class
Topic Author
Posts: 120
Joined: Thu Feb 19, 2015 2:23 am

Question about building a tiny .qcow2 for different uses

Sun Jul 23, 2017 7:14 pm

As a GNS3 user/contributor, I see a LOT of posts from people that are perplexed about why the changes they make to their docker containers vanish after they shut down their projects and restart it.

From my limited understanding of Docker, only things that were present in the dockerfile used during build-time will be persistent. Sure, there's a read/write part, where folks can install python, etc..., but those changes aren't permanent.

I've built qemu .qcow2 images before, that I had made for specific uses, like one for Ansible, one for ZTP, one for Puppet, a AAA server, and others, but folks have balked at downloading a 1-2GB image file. I even found a small 256MB debian wheezy .qcow2 image someone made, but it'll bloat up to 1-2GB, after all the dependencies are installed. Does anyone know of a linux distro that I could use to create similar server images, but something more in the 500MB? Once I figure out how to make docker containers, I could do something similar, but then folks would squawk about their Ansible playbooks, or other files going AWOL. Think I could do something similar with Arch or Alpine Linux?
 
blahsaysblah
Gerbil Elite
Posts: 581
Joined: Mon Oct 19, 2015 7:35 pm

Re: Question about building a tiny .qcow2 for different uses

Sun Jul 23, 2017 9:40 pm

Have you tried Alpine as your base image? Though it seems you need to cut down your dependencies and not the base image...
 
Doctor Venture
Gerbil First Class
Topic Author
Posts: 120
Joined: Thu Feb 19, 2015 2:23 am

Re: Question about building a tiny .qcow2 for different uses

Sun Jul 23, 2017 9:51 pm

blahsaysblah wrote:
Have you tried Alpine as your base image? Though it seems you need to cut down your dependencies and not the base image...



I just gave it a shot a few minutes ago. Ansible seemed to install with "apk add ansible", and the resulting image was only 326MB (I used the alpine-virt .iso image to make the .qcow2 disk image). I need to test it out, though, since I also installed nano, and it's not on there. I had to use vi to set up a static ip, and configure it to use a serial console, instead of needing VNC.
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: Question about building a tiny .qcow2 for different uses

Mon Jul 24, 2017 8:01 am

You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.
 
Doctor Venture
Gerbil First Class
Topic Author
Posts: 120
Joined: Thu Feb 19, 2015 2:23 am

Re: Question about building a tiny .qcow2 for different uses

Mon Jul 24, 2017 2:54 pm

notfred wrote:
You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.


I haven't done something like that in almost 17 years. I'll definitely need to either find my "Building Linux from the ground up" book, or a more recent edition, since a lot of this is waaaay out of my wheelhouse. I know just enough about Ubuntu, Debian, Fedora, WindRiver, and Montevista to be dangerous, and that's just because the last four are used in some of the network gear I install/provision.

I really wish Alcatel-Lucent/Nokia would make a trial version of VSR freely available for learning purposes. The last version of their TiMOS sim I have is 13.0R.6, and is really limited. I'd love to try even a bandwidth crippled version of VSR, just because I'd love to see what it can do. I've *seen* their 7750 Service Routers, but only from a distance. I wasn't allowed anywhere near them. :-?
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Question about building a tiny .qcow2 for different uses

Mon Jul 24, 2017 3:18 pm

notfred wrote:
You can always build up from nothing. Start with a blank file, run a partitioning tool on it and then loopback mount at the offset of the partition. You can then make a filesystem on it, mount that and put what you need on there, unmount everything and convert to a qcow2.

FWIW the kpartx tool is quite useful for doing stuff like this. It saves you from having to manually specify the partition offsets by automatically creating device entries for the partitions under /dev/mapper.
Nostalgia isn't what it used to be.
 
bthylafh
Maximum Gerbil
Posts: 4320
Joined: Mon Dec 29, 2003 11:55 pm
Location: Southwest Missouri, USA

Re: Question about building a tiny .qcow2 for different uses

Mon Jul 24, 2017 3:31 pm

The canonical way to create Linux from scratch is... Linux From Scratch.

http://www.linuxfromscratch.org/lfs/read.html

On a lark I did that about three weeks ago; took a few hours, most of which was waiting for glibc to compile, twice. I wouldn't suggest it for making a Linux you use on a daily basis - having a good dependency-resolving package manager is just too much of a convenience - but it's an interesting thing to try.
Hakkaa päälle!
i7-8700K|Asus Z-370 Pro|32GB DDR4|Asus Radeon RX-580|Samsung 960 EVO 1TB|1988 Model M||Logitech MX 518 & F310|Samsung C24FG70|Dell 2209WA|ATH-M50x
 
Doctor Venture
Gerbil First Class
Topic Author
Posts: 120
Joined: Thu Feb 19, 2015 2:23 am

Re: Question about building a tiny .qcow2 for different uses

Mon Jul 24, 2017 3:40 pm

bthylafh wrote:
The canonical way to create Linux from scratch is... Linux From Scratch.

http://www.linuxfromscratch.org/lfs/read.html

On a lark I did that about three weeks ago; took a few hours, most of which was waiting for glibc to compile, twice. I wouldn't suggest it for making a Linux you use on a daily basis - having a good dependency-resolving package manager is just too much of a convenience - but it's an interesting thing to try.


I'll check that out. Thanks! :D

I've been avoiding building ubuntu server VMs, due to the sheer bloat (that's why I'm hoping Alpine-virt pans out), but if Linux from Scratch pans out, that'll be cool, too.

I did try that canonical "automated" Openstack build process, but it always failed on one of the final steps. It was much simpler to just make an Openstack controller, and storage and compute nodes by hand (I was testing out a section of a data center topology I was building).
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: Question about building a tiny .qcow2 for different uses

Tue Jul 25, 2017 8:19 am

Doing something like Linux From Scratch teaches you a lot about how the system bootstraps up and gets running, very useful. I did something like that with my diskless folding thing that I had years ago. The source is still out there https://sourceforge.net/p/foldingcd/code/HEAD/tree/

And thanks JBI for the pointer to kpartx, that's a lot easier than calculating offsets!
 
rona rosen1
Gerbil In Training
Posts: 7
Joined: Sun Jun 04, 2017 1:35 am

Re: Question about building a tiny .qcow2 for different uses

Wed Nov 15, 2017 3:07 am

Docker containers do persist after exits, unless the
docker run 
command used the
--rm
option. So if you exit the shell, then restart and re-attach to that container the earlier files are still there. However, Docker management best practices are to consider containers as read-only and generate new images with a Dockerfile and the
docker build
command. While you are in a container, you can use the
docker commit 
command, which will save the container contents to a new image, it's just not recommended to use that as a way of "saving" your container.

There are ways to reduce docker image sizes, such as using a smaller base image, minimize layers, using the
no-install-recommends
option when running the
apt-get install
command. Docker has a lot of flexibility, but it's also a new way of thinking about computing so using it like a standard storage device can get you in trouble. Some best practices models for Docker have the data kept externally to the container, like with host volume mounts. There's a cool wiki that has all sorts of information about containers, and it has resources on how to create docker images if you are interested, the Container Technology Wiki

Apologies, I'm not sure why that "Code, Select All" weird formatting is in there. Sorry. I'm a noob.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On