Sign in Join
Wiki copytemporary

Qemu and other emulators: what changed

Revision 1 to 2. Removed lines have a minus sign; added lines have a plus sign.

OldNewLine
11This note covers setting up Qemu on an x86-based development system running Linux. This allows native (rather than cross) development tools to be run, which can be useful:
22
33- where the target system has performance/resource issues (e.g. some ARM systems)
44- is not run natively due to company policy (older versions of Microsoft Windows)
55- is quite simply unavailable at a reasonable price (e.g. SGI MIPS systems or the Chinese MIPS-based systems).
66
77It also briefly mentions User Mode Linux for x86 and the Hercules emulator for IBM zSeries mainframes despite the fact that these are not particularly relevant to Free Pascal, and Docker containerisation. It does not consider x86-on-x86 virtualisation systems such as VMware, Microsoft Hyper-V or Oracle VirtualBox, and only considers Linux KVM as a foundation technology for Qemu.
88
99## The Host System
1010
1111In the current case, the host is a Compaq rack-mount server running at around 3GHz with several Gb RAM; be warned that performance will drop off drastically with a lower specification system. It has two internal drive cages, the first is connected to a RAID controller and is used for the host operating system and tools, the second is connected to a SCSI controller and contains 6x discs each of which is used for a different guest system.
1212
1313The host IP address is 192.168.1.22 and the system is named pye-dev-07, the default gateway and name server are on 192.168.1.1. Guest systems are on the 192.168.22.x subnet and are named pye-dev-07a (192.168.22.16), pye-dev-07b (192.168.22.17) and so on, they have their own gateway 192.168.22.1 which is known to the site router and firewalls.
1414
1515The host operating system is Debian "Squeeze", the host normally runs headless and may be accessed by SSH, X using XDMCP, or VNC. The display manager is gdm since this has a better XDMCP implementation than the alternatives, however in practice graphical login is most often handled by VNC.
1616
1717The following guests are implemented:
1818
1919pye-dev-07a
2020Debian on ARM (little-endian, armel) using Qemu
2121
2222pye-dev-07b
2323Debian on MIPS (little-endian, mipsel) using Qemu
2424
2525pye-dev-07c
2626Slackware x86 13.37 using Qemu
2727
2828pye-dev-07d
2929Slackware x86 13.37 using User Mode Linux
3030
3131pye-dev-07e
3232Windows 2K using Qemu
3333
3434pye-dev-07f
3535Debian on zSeries using the Hercules emulator
3636
3737pye-dev-07g
3838Debian on 68k using the Aranym emulator
3939
4040Originally, pye-dev-07a was earmarked for big-endian ARM, but this appears to be being phased out by Debian so is probably no longer a viable target. Anybody planning to port FPC to the AVR-based Arduino?
4141
4242In general, multiple guests can run simultaneously although this has not been exhaustively tested recently.
4343
4444In the case of Linux the guest systems are each installed on an 18Gb disc, in the case of Windows a 36Gb disc is used. Each disc is assigned a label using e2label (arm, armel and so on), so that the startup script can mount it by name irrespective of which drive cage slot it's in.
4545
4646## Debian Guest using Qemu
4747
4848Select a suitable Debian mirror and version, for example
4949
5050` `[`http://ftp.de.debian.org/debian/dists/squeeze/main/`](http://ftp.de.debian.org/debian/dists/squeeze/main/)`...`
5151
5252Fetch a kernel and initrd image for Debian Squeeze, as below.
5353
54-<u>For ARM (little-endian):</u>
54+For ARM (little-endian):
5555
5656` .../main/installer-armel/current/images/versatile/netboot/vmlinux-2.6.32-5-versatile`\
5757` .../main/installer-armel/current/images/versatile/netboot/initrd.gz`
5858
5959In addition for this architecture you also need:
6060
6161` `[`http://people.debian.org/~aurel32/qemu/armel/initrd.img-2.6.32-5-versatile`](http://people.debian.org/~aurel32/qemu/armel/initrd.img-2.6.32-5-versatile)
6262
63-<u>For MIPS (little-endian):</u>
63+For MIPS (little-endian):
6464
6565`.../main/installer-mipsel/current/images/malta/netboot/vmlinux-2.6.32-5-4kc-malta`\
6666`.../main/installer-mipsel/current/images/malta/netboot/initrd.gz`
6767
6868Copy these to the disc reserved for the guest, e.g. /export/mipsel.
6969
7070Create a filesystem for Qemu, e.g.:
7171
7272``` bash
7373# qemu-img create -f qcow mipsel_hda.img 16G
7474```
7575
7676Expect that to round up to around 17.1 Gb, if it doesn't then experiment. Start Qemu, telling it what kernel, initrd and filesystem to use:
7777
78-<u>For ARM (little-endian):</u>
78+For ARM (little-endian):
7979
8080``` bash
8181# qemu-system-arm -M versatilepb -kernel vmlinuz-2.6.32-5-versatile -initrd initrd.gz \
8282-hda armel_hda.img -append root=/dev/ram
8383```
8484
8585Note that the above command requires X access, e.g. ssh with the -X option.
8686
87-<u>For MIPS (little-endian):</u>
87+For MIPS (little-endian):
8888
8989``` bash
9090# qemu-system-mipsel -M malta -kernel vmlinux-2.6.32-5-4kc-malta -initrd initrd.gz \
9191-hda mipsel_hda.img -append "root=/dev/ram console=ttyS0" -nographic
9292```
9393
9494Install the guest operating system as usual, splitting the disc into 16.5Gb for / with the remainder (around 700Mb) as swap. This will be slow, 8 or 9 hours is not implausible, so make sure that nobody's about to turn off your mains or disconnect you from the Internet.
9595
9696Don't worry if it tells you it's not installing a loader- it's not needed since the kernel and initrd are loaded into memory by the host.
9797
9898Boot the operating system and set network addresses etc. Use 192.168.22.16 or similar, with a gateway of 192.168.22.1.
9999
100-<u>For ARM (little-endian):</u>
100+For ARM (little-endian):
101101
102102``` bash
103103# qemu-system-arm -M versatilepb -kernel vmlinuz-2.6.32-5-versatile \
104104 -initrd initrd.img-2.6.32-5-versatile -hda armel_hda.img -append "root=/dev/sda1"
105105```
106106
107-<u>For MIPS (little-endian):</u>
107+For MIPS (little-endian):
108108
109109``` bash
110110# qemu-system-mipsel -M malta -kernel vmlinux-2.6.32-5-4kc-malta \
111111 -hda mipsel_hda.img -append "root=/dev/sda1 console=ttyS0" -nographic
112112```
113113
114114Finally, you should be able to boot the operating system with an operational network. This relies on having /etc/qemu-ifup and /etc/qemu-ifdown files (see below), and passes additional parameters to them in shell variables. In outline:
115115
116-<u>For ARM (little-endian):</u>
116+For ARM (little-endian):
117117
118118``` bash
119119# qemu-system-arm -M versatilepb -m 256 -hda armel_hda.img \
120120 -kernel vmlinuz-2.6.32-5-versatile -initrd initrd.img-2.6.32-5-versatile \
121121 -append 'root=/dev/sda1 text' \
122122 -net nic,macaddr=00:16:3e:00:00:01 -net tap,ifname=tun1
123123```
124124
125-<u>For MIPS (little-endian):</u>
125+For MIPS (little-endian):
126126
127127``` bash
128128# qemu-system-mipsel -M malta -m 256 -hda mipsel_hda.img \
129129 -kernel vmlinux-2.6.32-5-4kc-malta -no-reboot \
130130 -append 'root=/dev/sda1 console=ttyS0' -nographic \
131131 -net nic,macaddr=00:16:3e:00:00:02 -net tap,ifname=tun2
132132```
133133
134134Remember that if you change the network interface type or MAC address you will probably need to delete entries from the guest's /etc/udev/rules.d/z??_persistent-net.rules file.
135135
136136## Windows 2K Guest using Qemu
137137
138138Use dd to save a .iso image of the installation CD. Create a filesystem image:
139139
140140``` bash
141141# qemu-img create -f qcow2 win2k.img 32G
142142```
143143
144144Boot using startup script as below. Note that this must specify a non-default network card, since Qemu's current (as of 2011) default is not supported by Windows 2K.
145145
146146*TODO: run with kernel support module.*
147147
148148## KVM on a Linux host
149149
150150KVM (Kernel-based Virtual Machine) is an enabling API supported on more recent x86 and x86-64 (AMD64) systems, this replaces the older KQemu kernel module which is now deprecated by both Qemu and the kernel.
151151
152152KVM is typically enabled by the host system BIOS. If not enabled by the BIOS it cannot be enabled by the kernel or by (a suitably privileged) application program, since the x86 architecture requires power to be cycled to change the state of this facility. The result of this is that KVM might be unavailable (no /dev/kvm device) even if it shown as supported by the CPU flags in /proc/cpuinfo.
153153
154154## Common Qemu startup, ifup and ifdown scripts
155155
156156There is much commonality irrespective of whether the guest is running Linux or Windows.
157157
158-<u>First startup script (e.g. /export/C):</u>
158+First startup script (e.g. /export/C):
159159
160160``` bash
161161#!/bin/sh
162162
163163mount -L mipsel
164164cd /export/mipsel
165165. ./C-2
166166```
167167
168-<u>Second startup script for ARM (little-endian):</u>
168+Second startup script for ARM (little-endian):
169169
170170``` bash
171171#!/bin/sh
172172
173173# Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup
174174# to condition ARP, forwarding etc.
175175
176176QEMU_ID=1
177177QEMU='qemu-system-arm -M versatilepb'
178178QEMU_RAM='-m 256'
179179QEMU_HD='-hda armel_hda.img'
180180QEMU_CD=''
181181QEMU_BOOT="-kernel vmlinuz-2.6.32-5-versatile -append 'root=/dev/sda1 text' -initrd initrd.img-2.6.32-5-versatile"
182182# QEMU_MONITOR='-monitor stdio -nographic'
183183# QEMU_MONITOR='-nographic'
184184QEMU_VGA=''
185185VNC_ID=$(($QEMU_ID+1))
186186# QEMU_VNC="-vnc :$VNC_ID -k en-gb"
187187QEMU_VNC=''
188188QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID -net tap,ifname=tun$QEMU_ID"
189189QEMU_GUEST_IP_ADDRESS=192.168.22.17
190190QEMU_GUEST_IP_GATEWAY=192.168.22.1
191191QEMU_HOST_GATEWAY_IF=eth1
192192export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF
193193
194194echo * $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
195195 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC
196196
197197screen -S QEMU_$QEMU_ID \
198198sh -c "$QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
199199 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC"
200200
201201cd ..
202202```
203203
204-<u>Second startup script for MIPS (little-endian):</u>
204+Second startup script for MIPS (little-endian):
205205
206206``` bash
207207#!/bin/sh
208208
209209# Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup
210210# to condition ARP, forwarding etc.
211211
212212QEMU_ID=2
213213QEMU='qemu-system-mipsel -M malta'
214214QEMU_RAM='-m 256'
215215QEMU_HD='-hda mipsel_hda.img'
216216QEMU_CD=''
217217QEMU_BOOT="-kernel vmlinux-2.6.32-5-4kc-malta -append 'root=/dev/sda1 console=ttyS0' -no-reboot"
218218# QEMU_MONITOR='-monitor stdio -nographic'
219219QEMU_MONITOR='-nographic'
220220QEMU_VGA=''
221221VNC_ID=$(($QEMU_ID+1))
222222# QEMU_VNC="-vnc :$VNC_ID -k en-gb"
223223QEMU_VNC=''
224224QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID -net tap,ifname=tun$QEMU_ID"
225225QEMU_GUEST_IP_ADDRESS=192.168.22.18
226226QEMU_GUEST_IP_GATEWAY=192.168.22.1
227227QEMU_HOST_GATEWAY_IF=eth1
228228export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF
229229
230230echo * $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
231231 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC
232232
233233screen -S QEMU_$QEMU_ID \
234234sh -c "$QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
235235 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC"
236236
237237cd ..
238238```
239239
240-<u>Second startup script for Windows:</u>
240+Second startup script for Windows:
241241
242242``` bash
243243#!/bin/sh
244244
245245# Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup
246246# to condition ARP, forwarding etc.
247247
248248QEMU_ID=4
249249QEMU=qemu
250250QEMU_RAM='-m 256'
251251QEMU_HD='-hda win2k.img'
252252QEMU_CD='-cdrom Windows2k-SP4.iso'
253253QEMU_BOOT='-boot c'
254254QEMU_MONITOR='-monitor stdio'
255255QEMU_VGA='-vga cirrus'
256256VNC_ID=$(($QEMU_ID+1))
257257QEMU_VNC="-vnc :$VNC_ID -k en-gb"
258258QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID,model=rtl8139 -net tap,ifname=tun$QEMU_ID"
259259QEMU_GUEST_IP_ADDRESS=192.168.22.20
260260QEMU_GUEST_IP_GATEWAY=192.168.22.1
261261QEMU_HOST_GATEWAY_IF=eth1
262262export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF
263263
264264echo * $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
265265 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC
266266
267267screen -S QEMU_$QEMU_ID \
268268$QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \
269269 $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC
270270
271271cd ..
272272```
273273
274-<u>/etc/qemu-ifup (for both Linux and Windows):</u>
274+/etc/qemu-ifup (for both Linux and Windows):
275275
276276``` bash
277277#!/bin/bash
278278
279279# if-up file for qemu, heavily cribbed from the command sequence embedded in
280280# User Mode Linux. MarkMLl.
281281
282282echo Running /etc/qemu-ifup $1 $2...
283283
284284# For compatibility with UML the only parameter here is $1 which is the
285285# interface name. I've put in a reference to $2 so we can see it if anything
286286# changes.
287287
288288# I'm going to assume that qemu is always run by root. This is fairly
289289# reasonable since it allows guest OSes to be fired up which themselves might
290290# give access to confidential data etc. if compromised.
291291
292292# Here's my equivalent to the host-side UML setup for Qemu. We're hamstrung
293293# here by the fact that the emulator is not telling us what IP address it's
294294# trying to enable, there isn't a 1:1 correspondence between IP addresses and
295295# interfaces since the latter depends on the order the sessions are started.
296296#
297297# As a hack, assume that the caller exports QEMU_GUEST_IP_ADDRESS (e.g.
298298# 192.168.17.16), QEMU_GUEST_IP_GATEWAY (e.g. 192.168.17.1) and
299299# QEMU_HOST_GATEWAY_IF (e.g. eth0).
300300
301301echo * modprobe tun
302302modprobe tun
303303
304304echo * ifconfig $1 $QEMU_GUEST_IP_GATEWAY netmask 255.255.255.255 up
305305ifconfig $1 $QEMU_GUEST_IP_GATEWAY netmask 255.255.255.255 up
306306
307307X=`cat /proc/sys/net/ipv4/ip_forward`
308308if [ "$X" == "0" ]; then
309309
310310 # Use either this...
311311
312312# echo Global forwarding is not enabled. Please refer to the administrator
313313# echo responsible for this machine, enabling it might be a security hazard.
314314
315315 # ...or this.
316316
317317 echo Forcibly enabling global forwarding, note that this might be a security hazard.
318318 echo * echo 1 \> /proc/sys/net/ipv4/ip_forward
319319 echo 1 > /proc/sys/net/ipv4/ip_forward
320320 X=`cat /proc/sys/net/ipv4/ip_forward`
321321 if [ "$X" == "0" ]; then
322322 echo Unable to enable global forwarding. Please refer to the administrator
323323 echo responsible for this machine.
324324 fi
325325fi
326326
327327echo * route add -host $QEMU_GUEST_IP_ADDRESS dev $1
328328route add -host $QEMU_GUEST_IP_ADDRESS dev $1
329329
330330echo * echo 1 \> /proc/sys/net/ipv4/conf/$1/proxy_arp
331331echo 1 > /proc/sys/net/ipv4/conf/$1/proxy_arp
332332X=`cat /proc/sys/net/ipv4/conf/$1/proxy_arp`
333333if [ "$X" == "0" ]; then
334334 echo -n Retrying
335335 while [ "$X" == "0" ]; do
336336 sleep 1
337337 echo -n .
338338 echo 1 > /proc/sys/net/ipv4/conf/$1/proxy_arp
339339 X=`cat /proc/sys/net/ipv4/conf/$1/proxy_arp`
340340 done
341341 echo OK
342342fi
343343
344344echo * arp -Ds $QEMU_GUEST_IP_ADDRESS $1 pub
345345arp -Ds $QEMU_GUEST_IP_ADDRESS $1 pub
346346
347347echo * arp -Ds $QEMU_GUEST_IP_ADDRESS $QEMU_HOST_GATEWAY_IF pub
348348arp -Ds $QEMU_GUEST_IP_ADDRESS $QEMU_HOST_GATEWAY_IF pub
349349
350350# Set up experimental UDP proxies. Depending on the protocol of interest
351351# messages in one or both directions might need to be relayed.
352352#
353353# UDP port 79 is used for Dialarm signals, a unidirectional proxy is
354354# adequate for this but detection of hosts changing state (i.e. being
355355# added to or removed from the population of cooperating systems) is far
356356# more responsive if a bidirectional proxy is available.
357357
358358PROXY_ID=1
359359case "$1" in
360360 tun1)
361361 PROXY_ID=2
362362 ;;
363363 tun2)
364364 PROXY_ID=3
365365 ;;
366366 tun3)
367367 PROXY_ID=4
368368 ;;
369369 tun4)
370370 PROXY_ID=5
371371 ;;
372372 tun5)
373373 PROXY_ID=6
374374 ;;
375375 tun6)
376376 PROXY_ID=7
377377 ;;
378378 tun7)
379379 PROXY_ID=8
380380 ;;
381381esac
382382
383383# echo * udp-broadcast-relay -f $PROXY_ID 79 $QEMU_HOST_GATEWAY_IF $1
384384# /usr/local/src/udp-broadcast-relay/udp-broadcast-relay-0.3/udp-broadcast-relay \
385385# -f $PROXY_ID 79 $QEMU_HOST_GATEWAY_IF $1
386386
387387# Alternatively use this one which is oriented towards IP addresses
388388# rather than interfaces.
389389
390390# Note attempt to counteract any niceness applied to Qemu itself.
391391
392392ps ax | grep 'udp-proxy[ ]-z 79 ' >/dev/null 2>&1
393393if [ $? != 0 ]; then
394394 echo * udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS
395395 /usr/bin/nice --adjustment=20 /usr/local/src/udp-proxy/udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS
396396else
397397 echo * Already running udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS
398398fi
399399
400400# echo * udp-proxy -z 13264 $QEMU_GUEST_IP_ADDRESS
401401# /usr/local/src/udp-proxy/udp-proxy -z 13264 $QEMU_GUEST_IP_ADDRESS
402402
403403echo .../qemu/qemu-ifup completed.
404404```
405405
406-<u>/etc/qemu-ifdown (for both Linux and Windows):</u>
406+/etc/qemu-ifdown (for both Linux and Windows):
407407
408408``` bash
409409#!/bin/sh
410410
411411 echo * route del -host $QEMU_GUEST_IP_ADDRESS dev $1
412412route del -host $QEMU_GUEST_IP_ADDRESS dev $1
413413
414414echo * ifconfig $1 down
415415ifconfig $1 down
416416```
417417
418418In actual fact, these operations were cribbed from User Mode Linux (below) where they are embedded inside a host library.
419419
420420## Slackware x86 Guest using User Mode Linux
421421
422422User Mode Linux runs a guest kernel as a standard program, i.e. there is no emulation or virtualisation involved. The guest kernel can be allocated either physical discs or filesystems contained in files.
423423
424424Put a .iso corresponding to a recent Slackware DVD in /export/uml. Unpack the initrd using zcat and cpio, save it as an ext3 image initrd_unpacked. Using dd create an empty file root_fs_slackware which will be partitioned and formatted during installation.
425425
426426Use the sources from e.g. a recent Slackware to compile kernel plus modules with ARCH=um using a suffix -uml. Save the kernel to /export/uml/linux, install the modules and then copy them into the initrd filesystem.
427427
428428Boot the UML kernel, telling it to use the initrd image and DVD iso:
429429
430430``` bash
431431# ./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso rw
432432```
433433
434434Run fdisk and setup as normal, you might need to tell it to install to /dev/ubd1 and use /dev/ubd2 for source. Finally, copy the modules onto the target filesystem.
435435
436436When complete start up like this:
437437
438438``` bash
439439# Routine startup of a UML guest relies on (the host) running /usr/lib/uml/uml_net
440440# to condition ARP, forwarding etc.
441441
442442echo * ./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso \
443443 root=/dev/ubdb1 eth0=tuntap,,,192.168.1.22
444444
445445screen -S UML_3 \
446446./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso \
447447 root=/dev/ubdb1 eth0=tuntap,,,192.168.1.22
448448
449449cd ..
450450```
451451
452452Note that this is usually run from an X session, since the multiple virtual consoles appear as separate xterms. Also see [1](http://wiki.freepascal.org/linux/kernel/user_mode_VM).
453453
454454## Debian 68k Guest using Aranym
455455
456456``` bash
457457# apt-get install aranym
458458```
459459
460460I (MarkMLl) note this installs bridge-utils but instead am going to use my standard Qemu-style startup scripts, which themselves were originally based on sequences executed internally by UML; note that Hercules for zSeries (above) is the odd-one-out here since the guest uses SLIP for networking.
461461
462462Referring to <https://wiki.debian.org/Aranym/Quick>, download <https://www.freewrt.org/~tg/dc11m68k.tgz> and check its signature. Unpack, noting a kernel vmlinuz-2.6.39-2-atari, a filesystem image dc11.img.xz and a configuration file aranym.config.
463463
464464In aranym.config, change the ETH0 section to read:
465465
466466`Type = ptp`\
467467`HostIP = 192.168.22.1`\
468468`AtariIP = 192.168.22.22`\
469469`Netmask = 255.255.255.0`
470470
471471Change the startup script runaranym to read:
472472
473473``` bash
474474#!/bin/sh
475475
476476QEMU_GUEST_IP_ADDRESS=192.168.22.22
477477QEMU_GUEST_IP_GATEWAY=192.168.22.1
478478QEMU_HOST_GATEWAY=eth0
479479export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY
480480
481481/etc/qemu-ifup tap7
482482
483483cd "$(dirname "$0")"
484484SDL_AUDIODRIVER=dummy; export SDL_AUDIODRIVER
485485aranym-mmu -l -c aranym.config
486486
487487/etc/qemu-ifdown tap7
488488```
489489
490490Uncompress the image file:
491491
492492``` bash
493493# unxz dc11.img.xz
494494```
495495
496496Using xterm on a graphical login, run the startup script:
497497
498498``` bash
499499# runaranym
500500```
501501
502502The result of that should be a console for the guest Debian system.
503503
504504On the guest console, login as root with password root, and immediately change the password to something appropriate using the passwd command. Change the hostname in /etc/hostname and /etc/hosts, the IP address etc. in /etc/network/interfaces, and the nameserver in /etc/resolv.conf. Reboot and check that the network is operational by pinging from the guest to the site router (e.g. 192.168.1.1) and then pinging the guest (192.168.22.22) from any convenient system; if this doesn't work fix it before continuing.
505505
506506Then as described on <https://wiki.debian.org/Aranym/Quick> run the three commands:
507507
508508``` bash
509509# dpkg-reconfigure openssh-server
510510# apt-get update
511511# apt-get install popularity-contest
512512```
513513
514514Finally edit /root/profile to remove the reminder to run the above. It should now be possible to login using SSH, and to continue to configure and use the system like any Debian guest.
515515
516516## Debian zSeries Guest using Hercules, without VM
517517
518518Hercules is a commercial-grade emulator for IBM mainframes, it is available as a standard package for e.g. Debian and related Linux distributions. Once the emulator is running, enter
519519
520520`ipl 120`
521521
522522to boot Linux from device 120. Hopefully SSH will be operational so you won't need to interact with the console, but if you do then prefix each line that is to go to the guest operating system (i.e. rather than to the console itself) with a dot.
523523
524524Refer to the URL in the script below for more details.
525525
526526Startup:
527527
528528``` bash
529529#!/bin/sh
530530
531531# PREREQUISITE: Boot with ipl 120
532532
533533# Note that this makes no attempt to support IPv6.
534534
535535iptables -t nat -A POSTROUTING -o eth1 -s 192.168.22.0/24 -j MASQUERADE
536536iptables -A FORWARD -s 192.168.22.0/24 -j ACCEPT
537537iptables -A FORWARD -d 192.168.22.0/24 -j ACCEPT
538538echo 1 > /proc/sys/net/ipv4/ip_forward
539539echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp
540540
541541# http://www.josefsipek.net/docs/s390-linux/hercules-s390.html
542542
543543screen -S HERC_5 \
544544hercules
545545
546546cd ..
547547```
548548
549549Configuration:
550550
551551`CPUSERIAL 000069        # CPU serial number`\
552552`CPUMODEL  9672          # CPU model number`\
553553`MAINSIZE  256           # Main storage size in megabytes`\
554554`XPNDSIZE  0             # Expanded storage size in megabytes`\
555555`CNSLPORT  3270          # TCP port number to which consoles connect`\
556556`NUMCPU    2             # Number of CPUs`\
557557`LOADPARM  0120....      # IPL parameter`\
558558`OSTAILOR  LINUX         # OS tailoring`\
559559`PANRATE   SLOW          # Panel refresh rate (SLOW, FAST)`\
560560`ARCHMODE  ESAME         # Architecture mode ESA/390 or ESAME`\
561-\
561+
562562`# .-----------------------Device number`\
563563`# |     .-----------------Device type`\
564564`# |     |       .---------File name and parameters`\
565565`# |     |       |`\
566566`# V     V       V`\
567567`#---    ----    --------------------`\
568-\
568+
569569`# console`\
570570`001F    3270`\
571-\
571+
572572`# terminal`\
573573`0009    3215`\
574-\
574+
575575`# reader`\
576576`000C    3505    /export/zlinux/rdr/kernel.debian /export/zlinux/rdr/parmfile.debian /export/zlinux/rdr/initrd.debian autopad eof`\
577-\
577+
578578`# printer`\
579579`000E    1403    /export/zlinux/prt/print00e.txt crlf`\
580-\
580+
581581`# dasd`\
582582`0120    3390    /export/zlinux/dasd/3390.LINUX.0120`\
583583`0121    3390    /export/zlinux/dasd/3390.LINUX.0121`\
584-\
584+
585585`# tape`\
586586`0581    3420`\
587-\
587+
588588`# network                               s390     realbox`\
589589`# 0A00,0A01  CTCI -n /dev/net/tun -t 1500 10.1.1.2 10.1.1.1`\
590590`0A00,0A01  CTCI -n /dev/net/tun -t 1500 192.168.22.21 192.168.1.22`
591591
592592Note that the guest network is configured as SLIP via a simulated CTC (Channel To Channel) device. Best not fooled with.
593593
594594## Debian zSeries Guest using Hercules, with VM
595595
596596This combination is unlikely to work using freely-available software, since Linux requires at least an S/390 G3 system while the most recent IBM VM available is VM/370. It might be technically possible to run a non-free VM on Hercules, but at the time of writing it is at best unclear whether this could be done legally.
597597
598598This means that it is not, for example, possible to run a VM host with simultaneous Linux and MUSIC/SP guests.
599599
600600## MUSIC/SP using Sim/390 or Hercules
601601
602602*This section is an outline only. Note that in the IBM ecosystem OS and VM are the names of specific operating systems, rather than being a generic abbreviation; in addition DOS is a '60s-era IBM operating system with no connection to Microsoft or PCs.*
603603
604604MUSIC/SP is a freely-available (but not open source) operating system which implements a subset of IBM's MVS API, i.e. is to some extent compatible with operating systems of the OS/360 lineage in particular OS/VS1, extended with some novel features including a filesystem with user-accessible directories. It does not provide an API compatible with Linux, and uses the EBCDIC character set.
605605
606606Unlike other freely-available OS-compatible operating systems (see for example [2](http://www.ibiblio.org/jmaynard/), [3](http://gunkies.org/wiki/Installing_VM/370_on_Hercules), and the section below), MUSIC/SP provides TCP/IP networking. However this requires that the IUCV (Inter User Communication Vehicle) be provided by the underlying platform and that there is a suitable network support kernel for it to talk to: these are normally made available by IBM's VM operating system (VM/SP or VM/ESA, a sufficiently-recent version of which is not freely available).
607607
608608Considering emulated environments, IUCV can be provided either by running a recent VM on top of Hercules, or by running the freely-available (but not open source) Sim/390 emulator on Windows. Hercules does not provide IUCV or a network support kernel directly (although as of early 2012 this might be being addressed), so while MUSIC/SP will run on Hercules it will not have TCP/IP-based networking facilities: use Sim/390 if you really need this.
609609
610610Regrettably, the maintainer of MUSIC/SP and Sim/390 is no longer active, and while binaries and documentation remain available for download via [4](http://www.canpub.com/teammpg/de/mcgweb/) the sources are not.
611611
612612## VM/370 using Hercules
613613
614614*This section is an outline only. Note that in the IBM ecosystem OS and VM are the names of specific operating systems, rather than being a generic abbreviation; in addition DOS is a '60s-era IBM operating system with no connection to Microsoft or PCs.*
615615
616616VM/370, which is freely available as the "SixPack" (currently v1.2 as of early 2013), provides a hypervisor running on the "bare metal" which can host multiple single- or multitasking operating systems such as the (provided) Conversational Monitor System (CMS) or derivatives of DOS or OS (e.g. VSE or MVS).
617617
618618The CMS interactive environment is a distant cousin to unix, and is probably usable by anybody who remembers MS-DOS or older operating systems; the "SixPack" includes extensive but not complete online help. There is no networking API exposed to user programs: code and data may be moved between the host computer (running Hercules) and guest sessions by mounting files simulating tape devices, by simulations of an 80-column card punch and reader, or by a simulation of a line printer.
619619
620620In common with other IBM operating systems of the era, the documented API is in the form of macros for an IBM-compatible assembler; other types of interface are available including diag codes for communication between CMS and VM, and virtualised hardware access using Channel Control Words (CCWs). The services provided are roughly comparable with MS-DOS v1 or CP/M, i.e. there are separate sets of macros for handling different classes of peripherals: do not expect to open the terminal, card reader or printer as a file. Particularly if the S/380 hack (see below) is applied, the GCC compiler and standard libraries may be used but most software development and maintenance is done using assembler.
621621
622622## IBM S/370, S/390, and the S/380 "hack"
623623
624624*This section is an even sketchier outline.*
625625
626626As discussed elsewhere, the S/360 and S/370 architectures are limited to 24-bit addresses while the S/390 allows 31-bit addresses without, in general, breaking compatibility. There is an unofficial extension of the Hercules emulator (frowned upon by purists) that implements a non-standard "S/380" architecture and modifies the most-recent freely-available IBM mainframe operating systems (VSE nee DOS, MVS nee OS, and VM/370) to exploit this extension. Using this, there is sufficient available memory to run a large-scale compiler such as GCC *natively* on one of the classic IBM operating systems, with the important caveats that only one program can use this facility at a time (i.e. while a 31-bit GCC and a 24-bit make should work, two copies of the 31-bit GCC won't), and that in the case of VM one program means one program per computer rather than one program per virtualised system/login.
627627
628628To make use of this, you need the Hercules-380 patch from [5](http://mvs380.sourceforge.net/), a classic operating system such as the VM/CMS "sixpack" from [6](http://vm370.31bits.net/vm370sixpack-1_2.zip), and possibly the MECAFF enhancement for additional terminal sessions and the IND\$FILE program. In practice, it is impossible to do any of this without joining Yahoo!, and subscribing to the Hercules-VM370, hercules-os380 and H390-VM groups.
629629
630630## IA-64 (Itanium, Merced, McKinley etc.)
631631
632632*Terminally sketchy.*
633633
634634The FPC port targeting IA-64 exists in absolutely minimal form, i.e. a few skeleton files and that's about it. Since the IA-64 architecture appears to be heading in the same direction as its predecessor the iAPX-432, it's highly questionable whether any more work will be done on this architecture. However, for completeness (and because this author wants to be able to inspect some Itanium binaries)...
635635
636636While a few systems turn up on eBay etc., the asking price tends to be dictated by what the seller paid a few years ago rather than by what anybody expects to pay today. There is a simulator, written by HP, called "Ski", which is now open source. See [7](http://www.gelato.unsw.edu.au/IA64wiki/SkiSimulator), [8](http://www.hpl.hp.com/research/linux/ski/download.php) plus the Sourceforge project repository.
637637
638638(On a fairly complete development system running Debian) get the most recent Ski sources from Sourceforge. Run autogen.sh, installing e.g. autoconf, libtool and libelf-dev if necessary. Run make, install additional libraries such as gperf, bison, flex etc. as necessary. It might be necessary to edit syscall-linux.c to comment out the reference to asm/page.h which apparently isn't needed for very recent (Linux 3.x) kernels. On success get root and run make install.
639639
640640Assuming the disc image is a file named /export/ia-64/sda (e.g. renamed from the downloaded sda-debian) then
641641
642642`bski bootloader vmlinux simscsi=/export/ia-64/sd`
643643
644644Note the dropped final letter, that parameter is used as a prefix rather than being the name of a file or directory.
645645
646646As with a number of other emulators described on this page, this requires an X session for the console.
647647
648648This writer (MarkMLl) can't get networking running, either using the instructions at [9](http://www.hpl.hp.com/research/linux/ski/HOWTO-ski-networking.php) or using a tun/tap solution very roughly as suggested by [10](http://www.gelato.unsw.edu.au/IA64wiki/SkiVirtualNetwork). It might be necessary to try a much older host kernel, i.e. late 2.4 or early 2.6, however this is not currently being pursued due to lack of relevance.
649649
650650The kernel from HP is 2.4.20, the filesystem is described as "Debian Sid" but from the kernel version is probably somewhere between "Woody" (3.0) and "Sarge" (3.1). I don't know how easy it's going to be to get this up to scratch, at the very least a working network is going to be a prerequisite.
651651
652652------------------------------------------------------------------------
653653
654654## Relative Performance
655655
656656Performance is, in general, disappointing. Using an informal "torture test" which exercises the CPU and disc access:
657657
658658`Compaq ProLiant ML530 G2, 2.8GHz, 3Gb, 8 jobs, 390W 0m12.170         79`\
659-\
659+
660660`Linksys NSLU2, 266MHz 32Mb, 1 job, 7W               6m35.014s        46`\
661-\
661+
662662`Qemu ARM, 1 job, 390W                              42m58.925s    16,757`\
663-\
663+
664664`Qemu MIPS, 1 job, 390W                             17m49.103s     6,949`\
665-\
665+
666666`Qemu x86, 1 job, 390W                              47m0.441s     18,330`\
667-\
667+
668668`UML, 1 job, 390W                                    8m26.529s     3,289`\
669-\
669+
670670`Hercules zSeries, 4 jobs, 390W                      9m43.330s     3,790`
671671
672672The final column in the above list is the W-Mins to complete the job. These timings are without the benefit of kernel support from kqemu (obsolete) or KVM, but the fact that Qemu's support for MIPS is significantly better than that for other architectures, and the fact that the Hercules emulator is in all cases better than Qemu, does make one wonder how efficient the code is.
673673
674674Compiling FPC 2.4.2 using time make NOGDB=1 OPT='-O- -gl' all :
675675
676676`Host (x86, Linux, Debian):`\
677-\
677+
678678`  real    4m47.842s`\
679679`  user    3m42.126s`\
680680`  sys     0m33.506s`\
681-\
681+
682682`Slug (32Mb, ARM, Linux, Debian):`\
683-\
683+
684684`  real    284m58.543s`\
685685`  user    86m45.570s`\
686686`  sys     20m46.500s`\
687-\
687+
688688`Qemu (ARM, Linux, Debian):`\
689-\
689+
690690`  real    406m31.931s`\
691691`  user    236m49.030s`\
692692`  sys     148m58.110s`\
693-\
693+
694694`Qemu (x86, Linux, Slackware):`\
695-\
695+
696696`  real    141m45.700s`\
697697`  user    122m40.724s`\
698698`  sys     17m15.670s`\
699-\
699+
700700`Qemu (x86, Windows 2000):`\
701-\
701+
702702`  Elapsed 108m`\
703-\
703+
704704`UML (x86, Linux, Slackware):`\
705-\
705+
706706`  real    238m41.257s`\
707707`  user    45m54.460s`\
708708`  sys     3m44.140s`
709709
710710Compiling Lazarus 0.9.30 using time make LCL_PLATFORM=gtk2 bigide :
711711
712712`Host (x86, Linux, Debian):`\
713-\
713+
714714`  real    2m21.072s`\
715715`  user    2m6.452s`\
716716`  sys     0m12.285s`\
717-\
717+
718718`Slug (32Mb, ARM, Linux, Debian):`\
719-\
719+
720720`  real    9385m49.319s`\
721721`  user    67m23.460s`\
722722`  sys     430m9.870s`\
723-\
723+
724724`Qemu (ARM, Linux, Debian):`\
725-\
725+
726726`  real    281m55.536s`\
727727`  user    153m3.150s`\
728728`  sys     53m27.470s`\
729-\
729+
730730`Qemu (x86, Linux, Slackware):`\
731-\
731+
732732`  real    70m53.957s`\
733733`  user    60m3.474s`\
734734`  sys     8m8.801s`\
735-\
735+
736736`Qemu (x86, Windows 2000, default platform):`\
737-\
737+
738738`  Elapsed 53m`\
739-\
739+
740740`UML (x86, Linux, Slackware):`\
741-\
741+
742742`  real    489m40.233s`\
743743`  user    81m43.740s`\
744744`  sys     7m51.280s`
745745
746746## Graphical Access using VNC etc.
747747
748748All of the above assumes that SSH is available so that shell (command line) sessions can be run over the network. Where necessary install e.g. the Debian ssh (or openssh, openssh-server etc.) package. It is also possible to run a program using a GUI over SSH by X forwarding, see the SSH client's -X option and the corresponding server configuration-file settings.
749749
750750There are several ways of running a program using a GUI without forwarding the X protocol over SSH:
751751
752752- Running Qemu or UML in an xterm.
753753
754754- Using Qemu's host VNC support.
755755
756756- Connecting to a Qemu (or UML etc.) guest using X.
757757
758758- Connecting to a Qemu (or UML etc.) guest using VNC.
759759
760760In all cases it is worth using a low-resource window manager such as FluxBox or xfwm rather than KDE or Gnome.
761761
762762### Running Qemu or UML in an xterm or with host VNC support
763763
764764This depends on the build options of the guest system kernel etc. In general, display resolution and depth will be limited by the emulated hardware, the network ports are associated with the host (rather than guest) system. See the Qemu documentation for details.
765765
766766### Connecting to a Qemu (or UML etc.) guest using X
767767
768768Install the Gnome Display Manager (gdm) on the guest system. This allows a desktop system to log into it using XDMCP, the desktop system determines the display resolution and depth.
769769
770770### Connecting to a Qemu (or UML etc.) guest using VNC
771771
772772This method also works well for arbitrary systems, even if Qemu etc. is not being used.
773773
774774Install gdm and RealVNC (or, with limitations, TightVNC) on the guest system; the latter might be needed for ARM guests but RealVNC is to be preferred when available. This allows a desktop system to log into it using VNC, the guest system determines the display resolution and depth and network ports are associated with the guest (rather than host) system.
775775
776776In /etc/inittab, insert something like this:
777777
778778`6:23:respawn:/sbin/getty 38400 tty6`\
779-\
779+
780780`# Added in lieu of trying to get it working in inetd.conf. MarkMLl.`\
781-\
781+
782782`8:23:respawn:/usr/local/vnc/vncshim-0-20`\
783783`# 9:23:respawn:/usr/local/vnc/vncshim-4-24`\
784-\
784+
785785`# Example how to put a getty on a serial line (for a terminal)`\
786786`#`\
787787`#T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100`
788788
789789Many systems limit the number of VNC servers that may be run per system (including localhost) as the side effect of a window manager XDMCP security precaution. If necessary, edit the window manager configuration file (gdm.conf, kdmrc etc.) to increase the number of sessions that each XDMCP client can request, e.g. in the case of gdm.conf:
790790
791791`[xdmcp]`\
792792`Enable=true`\
793793`DisplaysPerHost=6`
794794
795795Create /usr/local/vnc/vncshim-0-20 to read:
796796
797797``` bash
798798#!/bin/bash
799799
800800# This forms a shim between /etc/inittab and the VNC server, necessitated by
801801# the facts that (a) I can't get this command working reliably in inetd.conf
802802# and (b) lines in initab have restricted length.
803803#
804804# Note that the name of the file reflects the fact that it might request non-
805805# standard combinations of VNC port and X display number. MarkMLl.
806806
807807# My preference is to select default linear dimensions that appear somewhere
808808# in the list of conventional graphics modes. 1200x960 fits inside a 1280x1024
809809# screen, even allowing that the title bar etc. will take additional space.
810810
811811SIZE=1200x960
812812
813813# The DPI setting can usefully be set to match the user's normal screen, leave
814814# undefined (commented out) if in doubt.
815815
816816DPI='-dpi 96'
817817
818818# Each VNC server running on a host must have a distinct suffix to prevent port
819819# and display number clashes.
820820
821821SUFFIX=0
822822
823823# Some option formats vary between RealVNC (top) and TightVNC (bottom).
824824
825825OPTIONS='DisconnectClients=0 -NeverShared'
826826# OPTIONS='-dontdisconnect -nevershared'
827827
828828####################### CHANGE NOTHING BELOW HERE ###########################
829829
830830VNCPORT=570$SUFFIX
831831XDISPLAY=:2$SUFFIX
832832
833833# Try to work out what this system is. Note particular care to allow spaces
834834# in the desktop name.
835835
836836WHATAMI=`dirname $0`/whatami
837837if [ -x $WHATAMI ]; then
838838 WHATAMI2=`$WHATAMI`
839839else
840840 if [ -r $WHATAMI ]; then
841841 WHATAMI2=`cat $WHATAMI`
842842 else
843843 WHATAMI2=$$
844844 fi
845845fi
846846DESKTOP='-desktop '\'$WHATAMI2\'
847847
848848# Optional Java viewer.
849849
850850if [ -r `dirname $0`/classes/index.vnc ]; then
851851 JAVA="-httpd `dirname $0`/classes -httpPort 580$SUFFIX"
852852fi
853853
854854# This is the only way I can get it working the way I want, i.e. with reliable
855855# handling of spaces in the desktop name. I'm hardly proud of it.
856856
857857rm -f `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX
858858
859859cat >`dirname $0`/.vncshim-$SUFFIX-1$SUFFIX <<EOT
860860#!/bin/sh
861861exec /usr/bin/Xvnc -geometry $SIZE $DPI -depth 16 $DESKTOP \
862862 -rfbport $VNCPORT -rfbauth /root/.vnc/passwd-$SUFFIX $JAVA \
863863 $OPTIONS -query localhost -once $XDISPLAY
864864EOT
865865
866866chmod +x `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX
867867exec `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX
868868```
869869
870870Create /usr/local/vnc/whatami to read:
871871
872872``` bash
873873#!/bin/bash
874874
875875echo -n "`hostname`, "
876876_WHATAMI=`uname -m`
877877if [[ $_WHATAMI == i?86 ]]; then
878878 _WHATAMI='i586'
879879fi
880880if [[ $_WHATAMI == arm* ]]; then
881881 _WHATAMI='arm'
882882fi
883883
884884if [ -x `dirname $0`/whatami-$_WHATAMI ]; then
885885 `dirname $0`/whatami-$_WHATAMI
886886else
887887 uname -m
888888fi
889889```
890890
891891Additional scripts e.g. whatami-arm can query /proc/cpuinfo, which is different for each architecture. Make sure that all scripts are set +x for root.
892892
893893Use vncpasswd to set up password files /root/.vnc/passwd-0 etc.
894894
895895So far, I find that RealVNC works fine when invoked as above, but TightVNC doesn't correctly invoke the display manager.
896896
897897## Accessing "Unusual" Devices
898898
899899This section covers devices such as audio (i.e. used for playing .wav files) and MIDI, and USB devices which are not claimed by the host or guest kernel.
900900
901901### Sound/MIDI for Qemu
902902
903903Don't expect this to be hi-fi quality, but it should be adequate for operator alerts etc.
904904
905905#### Identifying the Host's Hardware
906906
907907Obviously, a good starting point for this is using lspci and lsusb to determine what physical hardware is installed. The next stage is usually to consult dmesg output and to use lsmod to determine what subsystems the kernel has loaded hence what APIs are available. Graphical tools such as the KDE Info Centre can also be useful here, however identifying which API is most useful can be a bit of a black art.
908908
909909The current writer (MarkMLl) has a Debian "Squeeze" host with USB-connected audio and MIDI devices, with the latter connected to a Yamaha tone generator. An alternative (and more conventional) configuration would be to have an internal soundcard, with both audio and MIDI output feeding a loudspeaker.
910910
911911Knowing what hardware and subsystems are available, the remaining question is which subsystems Qemu can make use of. Running this command goes some way towards providing an answer:
912912
913913`$ qemu -audio-help |grep ^Name:`\
914914`Name: alsa`\
915915`Name: oss`\
916916`Name: sdl`\
917917`Name: esd`\
918918`Name: pa`\
919919`Name: none`\
920920`Name: wav`
921921
922922For reasons that are unclear *(roughly translated: I'd like some help here)* I've had most success with OSS, starting Qemu with a script that includes these lines:
923923
924924``` bash
925925export QEMU_AUDIO_DRV=oss
926926export QEMU_OSS_DAC_DEV=/dev/dsp1
927927export QEMU_OSS_ADC_DEV=/dev/dsp1
928928```
929929
930930#### Selecting the Guest's Soundcard
931931
932932Assuming that the guest is to have both audio and MIDI, common sense would suggest that the guest operating system should see both audio and MIDI hardware implemented by Qemu. According to <http://rubenerd.com/qemu-ad-lib-midi-win-31/>, this implies that Qemu should be recompiled with the capability of emulating an Adlib card, which for current versions of Qemu means running something like this:
933933
934934``` bash
935935make clean distclean
936936./configure --audio-card-list=ac97,es1370,sb16,adlib
937937make
938938```
939939
940940Irrespective of whether a custom Qemu is built or not, it's useful to check what devices it emulates:
941941
942942``` bash
943943$ qemu -soundhw ?
944944Valid sound card names (comma separated):
945945pcspk PC speaker
946946sb16 Creative Sound Blaster 16
947947ac97 Intel 82801AA AC97 Audio
948948es1370 ENSONIQ AudioPCI ES1370
949949```
950950
951951For some guest operating systems (e.g. Windows NT 4), adding Adlib emulation to Qemu is sufficient to get MIDI working with the Sound Blaster, i.e.
952952
953953``` bash
954954$ qemu ... -soundhw sb16,adlib ...
955955```
956956
957957In this case the guest uses only the standard Sound Blaster driver, possibly ignoring MPU-401 support.
958958
959959For guest systems which don't benefit from this, there's little point adding something like an Adlib card if the guest operating system doesn't have a driver for it- which appears to be the case with Windows 2000. The current writer's preference is to select the ES13770, since as a straightforward PCI device it should be enumerated without difficulty by almost any guest operating system:
960960
961961``` bash
962962$ qemu ... -soundhw es1370 ...
963963```
964964
965965Having started Qemu, it's possible to query the emulated devices:
966966
967967`(qemu) info pci`\
968968`...`\
969969`  Bus  0, device   4, function 0:`\
970970`    Audio controller: PCI device 1274:5000`\
971971`      IRQ 11.`\
972972`      BAR0: I/O at 0xc200 [0xc2ff].`\
973973`      id ""`
974974
975975The expected result of this is that the guest operating system would see audio hardware, but not MIDI. However in the case of Windows 2000 a MIDI device is emulated, so programs which use MIDI for e.g. operator alerts will work properly.
976976
977977*Anybody: need help here generalising this to other guest operating systems.*
978978
979979### USB for Qemu
980980
981981As an example, a Velleman K8055 board is plugged into the host. First use lsusb to gets its vid:pid identifiers:
982982
983983``` bash
984984$ lsusb
985985Bus 001 Device 005: ID 10cf:5500 Velleman Components, Inc. 8055 Experiment Interface Board (address=0)
986986```
987987
988988Now add these parameters to the Qemu command line:
989989
990990``` bash
991991qemu ... -usb -usbdevice host:10cf:5500 ...
992992```
993993
994994On the guest, lsusb should show the device available for use.
995995
996996The Velleman board is unusual in that it has ID jumpers which set the last digit, i.e. multiple devices appear as 10cf:5500 through 10cf:5503. In cases where this facility is not available, the bus and device number can be used:
997997
998998``` bash
999999qemu ... -usb -usbdevice host:1.5 ...
10001000```
10011001
10021002Obviously, there's a risk here that devices will move around between invocations.
10031003
10041004### USB for UML
10051005
10061006There was supposedly a patch that mapped unused USB devices on the host into the guest. I'm not sure it made it into the standard release.
10071007
10081008## QEMU User Emulation Mode in Chrooted Environment
10091009
10101010In previous chapters QEMU's full system emulation mode was described: a complete guest system is emulated including bios and hardware. Although this approaches a real environment as close as possible, the overhead created by emulating a complete system is considerable. QEMU's user emulation mode allows to run "guest" code directly on the "host" system. The cpu is still emulated but system calls are translated into system calls on the host system. Libraries used by the "guest" code are the "guest" libraries, not the host libraries. To run a small program created for fe. ARM with no dependencies you can simply do a
10111011
10121012`$ qemu-arm program`
10131013
10141014When the program has more and more dependencies, it becomes increasingly difficult to put all dependencies in locations that can be found by qemu-arm without messing up your host system. The solution that will be developed here is to create a chroot environment on the host in which all code is "guest" code. Advantage of this solution:
10151015
10161016- the chroot isolates the guest from the host
10171017- no need for vnc, ssh or any other solution to export screens.
10181018- no booting of a guest OS system.
10191019- guest "immersion" for user programs is almost complete. Only very low level and hardware related programming will notice the difference.
10201020- a spectacular speed improvement for graphics (X) related tasks (running lazarus f.e.)
10211021- hard disk resources are common to host and guest. Host can read and modify "guest" files but not the other way around.
10221022
1023-<u>Setting up the system.</u>
1023+Setting up the system.
10241024
10251025For a better comprehension of what follows, here is the directory structure used in this how-to:
10261026
10271027`home`\
10281028` qemu`\
10291029`   arm-linux-user`\
10301030`   sparc-linux-user`\
10311031`   scripts `\
10321032`   disks       // here are qemu images for the different systems emulated`\
10331033`   chroot`\
10341034`       mount   `\
10351035`       arm // here we'll chroot the arm environment`\
10361036`       sparc   `
10371037
10381038Get and build the latest qemu
10391039
10401040``` bash
10411041$ cd
10421042$ git clone git://git.savannah.nongnu.org/qemu.git qemu
10431043$ cd qemu
10441044```
10451045
10461046Build all emulations (full system and user emulation mode, all processors)
10471047
10481048``` bash
10491049$ ./configure --static to build
10501050```
10511051
10521052or only the user emulations needed
10531053
10541054``` bash
10551055$ ./configure --static --target-list=sparc-linux-user,arm-linux-user
10561056
10571057$ make clean all
10581058```
10591059
10601060The --static option makes that qemu-arm and qemu-sparc are linked statically. Since they will be running in the chroot environment later on, they must be build without dependencies. Some distributions provide already statically linked binaries. fe qemu-arm-static on debian. Although they are not the latest and greatest, using these packages makes it somewhat easier. Building QEMU from source is really just a matter of minutes.
10611061
10621062Now we need to copy the complete guest system into chroot/arm or chroot/sparc. There are several ways of doing this but when you don't have a real arm or sparc system at hand, using a QEMU image of a full system emulation as described in previous chapters is the easiest.
10631063
10641064Mount the QEMU image.
10651065
10661066WARNING: do not mount a qemu image when the associated guest is running!
10671067
10681068Only raw QEMU images can be mounted as a loop-back device. If you have a qcow formatted image, convert it to a raw image.
10691069
10701070``` bash
10711071$ qemu-img convert disks/armel_hda.img -O raw disks/armel_hda.raw
10721072
10731073$ cd chroot
10741074$ su
10751075# mount ../disks/armel_hda.raw mount -o loop,offset=32256
10761076```
10771077
10781078The offset is the start of the first partition on the ARM disk when the default disk layout was used during installation. If mount fails then the offset is probably wrong. To calculate the offset:
10791079
10801080``` bash
10811081# fdisk -lu ../disks/armel_hda.raw
10821082```
10831083
10841084Find the first sector of the partition that will be mounted as root. Calculate offset as (start sector number * 512). For the default SPARC debian etch installation using a sun partition table with a /boot first partition the offset is 98703360.
10851085
10861086Check if the correct partition is mounted:
10871087
10881088``` bash
10891089# ls mount
10901090```
10911091
10921092Copy the complete system
10931093
10941094``` bash
10951095# cp -r mount/* arm/
10961096# umount mount
10971097```
10981098
10991099Copy the QEMU arm emulator to the arm directory
11001100
11011101``` bash
11021102# cd arm
11031103# cp ../../arm-linux-user/qemu-arm usr/local/bin
11041104```
11051105
11061106Our "guest" system is complete. Since we are sharing our "host" devices and system, we need to mount a few special directories:
11071107
11081108``` bash
11091109# mount --bind /proc proc
11101110# mount --bind /dev dev
11111111# mount --bind /sys sys
11121112# mount --bind /tmp tmp
11131113```
11141114
11151115Why /tmp? The /tmp directory is used by the X window system. When you don't mount tmp on the host /tmp, you will get "can't open display 0:0" errors when trying to open a window in the guest.
11161116
11171117We are going to use the host network:
11181118
11191119``` bash
11201120# sudo cp /etc/resolv.conf etc
11211121```
11221122
11231123Remains one last thing: the emulator is copied into the guest but how do we tell the system that the emulator needs to kick in when a "quest" elf is detected? This is done with binfmt_misc. A configuration script is included with QEMU that registers the QEMU user mode emulators with binfmt_misc: (on my system I had to do a chmod +x on the script)
11241124
11251125``` bash
11261126# ../../scripts/qemu-binfmt-conf.sh
11271127```
11281128
11291129When you look at this script at the end of the binary data you'll notice that the emulator is registered as /usr/local/bin/qemu-arm. And that is where we put it a few lines ago.
11301130
11311131Launch the system:
11321132
11331133``` bash
11341134# chroot .
11351135```
11361136
11371137If all goes well:
11381138
11391139``` bash
11401140# uname -m
11411141armv7l
11421142```
11431143
11441144Everything you do from here is in your "guest" system. To exit the "guest" system.
11451145
11461146``` bash
11471147# exit
11481148```
11491149
11501150And unmount the special directories.
11511151
11521152``` bash
11531153# umount proc
11541154# umount dev
11551155# umount sys
11561156# umount tmp
11571157```
11581158
1159-<u>Conclusion.</u>
1159+Conclusion.
11601160
11611161The ARM emulation works very well, except for gdb. On the contrary there are still quite some problems with the SPARC emulation. There is a bug in duplicating pipes which makes the use of bash impossible. The sh shell works but several unix commands don't function correctly. What works fine is fpc and lazarus! Programs can't be launched from inside lazarus. Because of the improved graphics speed when using the user emulation mode it is still worthwhile for using lazarus. Instead of starting a shell in chroot, you start directly lazarus:
11621162
11631163``` bash
11641164$ sudo chroot . /your/lazarus/dir/lazarus
11651165```
11661166
11671167To run the program compiled in lazarus, you can launch it from another "host" terminal:
11681168
11691169``` bash
11701170$ cd qemu/chroot/arm
11711171$ sudo chroot . /your/program/dir/project
11721172```
11731173
11741174Here also gdb is not working.
11751175
11761176## Docker images
11771177
11781178Docker uses various APIs provided by the underlying operating system to isolate programs in separate namespaces.
11791179
11801180The implication of this is that every Docker container has its own process space (i.e. processes in different containers might have the same numeric PID), network space (processes in different containers can have different routing, firewall rules and so on), filesystems (except that they share kernel files and a number of crucial libraries and configuration files) and so on.
11811181
11821182Containerisation is distinct from virtualisation in that all containers which are running on the same host computer make use of the same host kernel, it is not possible for one container to use a different kernel (e.g. for development or deployment test purposes), and there is no absolute guarantee that a container taken over by malicious code cannot subvert the remainder of the system.
11831183
11841184***Note that there are pitfalls for the unwary. For example, having Docker installed and the Docker daemon running can break Qemu networking if this expects to have bridged access to the host interface rather than to be sitting on its own private subnet.***
11851185
11861186One thing that does work however, is running a 32-bit container on a 64-bit host, and in most cases the Linux kernel APIs are sufficiently backwards-compatible that the programs and libraries from an older release of a distro can run on a much more recent host. One of the cases which will be discussed here runs a container containing 32-bit Debian "Lenny" from 2008 on a host running Debian "Buster" from 2019, this combination allows e.g. compatibility with the GTK v1 widget set (no longer supported by more recent distreaux) to be tested.
11871187
11881188### Docker fundamentals
11891189
11901190This is not the place for a full discussion of the care and feeding of Docker, but it's worth establishing a few basic concepts.
11911191
11921192- Host: the physical computer running e.g. Qemu (with or without KVM) or Docker.
11931193
11941194- Guest: on a virtualised system (Qemu etc.), a completely separate memory space with its own kernel.
11951195
11961196- Container: on a Docker based system, a group of application programs sharing the host kernel but largely isolated from the rest of the host system using various namespace mechanisms.
11971197
11981198Looking specifically at Docker:
11991199
12001200- Dockerfile: a succinct description of what is to go into a Docker image.
12011201
12021202- Image: the Docker analogue of a bootable filesystem, initially built with reference to a Dockerfile.
12031203
12041204- Container: an image which has been run, whether or not it is currently running.
12051205
12061206Once running, a container has an associated parameter list which specifies the main program together with any sockets etc. exposed to the host. Neither an image nor a container correspond to a single file on the host, but:
12071207
12081208- A container may be stopped and committed to an image.
12091209
12101210- An image may be saved to a single file, in which state it can be moved between hosts.
12111211
12121212- An image may be run a set of parameters different from those of the container committed to it.
12131213
12141214The result of this is that if something is installed in the context of a container (e.g. Apache is added from the operating system's main repository) the container should be stopped, committed to a new image and then run with an additional parameter specifying what sockets etc. should be exposed. There are ways around that, but that combination appears to be the easiest to manage.
12151215
12161216The Docker community has a master repository, and the docker program on a host computer may search it to locate images. However in the context of those images there appears to be no "official" way of searching the tags that allow a specific version to be pulled, hence something "messy" and subject to change like
12171217
12181218 $ docker search debian
12191219
12201220 $ wget -q https://registry.hub.docker.com/v1/repositories/debian/tags -O - \
12211221 | sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | tr '}' '\n' | awk -F: '{print $3}' | grep buster
12221222
12231223 $ docker pull debian:buster
12241224
12251225or alternatively (for the really hard to reach bits)
12261226
12271227 $ docker search vicamo
12281228
12291229 $ wget -q https://registry.hub.docker.com/v1/repositories/vicamo/debian/tags -O - \
12301230 | sed -e 's/[][]//g' -e 's/"//g' -e 's/ //g' | tr '}' '\n' | awk -F: '{print $3}'
12311231
12321232 ...
12331233 buster-i386
12341234 buster-i386-sbuild
12351235 buster-i386-slim
12361236 ...
12371237
12381238 $ docker pull vicamo/debian:buster-i386-slim
12391239
12401240Generally speaking, if a developer on e.g. Github says that he has a special-purpose image, some combination of the above commands will allow the required version (e.g. in the above example tagged buster-i386-slim) to be pulled.
12411241
12421242Neither a container nor an image is directly mountable e.g. by using losetup. It is possible to copy files to or from a container using the `docker container cp` command, subject to the same precautions that one would take on a running unix system.
12431243
12441244### A 64-bit Docker container with basic utilities
12451245
12461246This is a basic installation of a 64-bit Debian stable release ("Buster" at the time of writing) on the same host version. This is useful in a situation where a developer wants to experiment with libraries etc. that he does not normally have installed, it should probably not be trusted to quarantine untrusted code.
12471247
12481248In the instructions below, commands on the host are run in the context of a normal user so are shown prefixed by a \$ prompt, while commands run in the context of the container are run by root so shown prefixed by a \# prompt.
12491249
12501250 $ docker pull debian:buster
12511251
12521252 $ docker run --interactive --tty debian:buster
12531253
12541254 # apt-get update
12551255
12561256 # apt-get install dialog tasksel python3
12571257
12581258 # tasksel --new-install
12591259
12601260 # dpkg-reconfigure locales
12611261
12621262Using tasksel, remove desktop and print server and add SSH. The author favours installing net-tools so that the `netstat -an` command can be used to check socket accessibility.
12631263
12641264 # apt-get install net-tools elvis-tiny
12651265
12661266So far there is no provision for starting the SSH daemon. Add or edit /etc/rc.local to contain
12671267
12681268 #!/bin/sh
12691269
12701270 rm -rf /tmp/*
12711271
12721272 # For cases where a 32-bit container is running (by implication on a 64-bit host), make
12731273 # sure that commands such as uname -a return consistent information to avoid breaking
12741274 # the traditional ./configure && make operation.
12751275
12761276 (file -L /bin/sh | grep -q 'ELF 32-bit LSB')
12771277 if [ $? -eq 0 ] ; then
12781278 LINUX32=linux32
12791279 fi
12801280
12811281 cd /etc/init.d
12821282 $LINUX32 ./ssh start
12831283 # $LINUX32 ./openbsd-inetd start
12841284
12851285 cd
12861286 exec $LINUX32 /bin/bash
12871287
12881288The ID of the current container is in the prompt. Exit it and commit to a new image, then start with a parameter to expose the SSH socket:
12891289
12901290 # exit
12911291
12921292 $ docker container commit 4efab4804fb6 debian_minimal
12931293
12941294 $ docker run -it -p 65022:22 \
12951295 --entrypoint=/etc/rc.local debian_minimal
12961296
12971297 # groupadd -g 1000 markMLl
12981298
12991299 # useradd -g markMLl -u 1000 -m -d /home/markMLl markMLl
13001300
13011301 # passwd markMLl
13021302
13031303Add newly created user name to sudo group (/etc/group and /etc/gshadow files). Note the container ID, if it is now stopped the parameters are preserved and it should be possible to start it in the background and connect as the above user via SSH.
13041304
13051305 $ docker container stop c5be7fd96866
13061306
13071307 $ docker container start c5be7fd96866
13081308
13091309 $ ssh localhost -p 65022
13101310
13111311The above is a good starting point for a general-purpose containerised system. The section below will also show how to get a graphical program (e.g. the Lazarus IDE) running in a container.
13121312
13131313### A 32-bit Docker container with FPC and Lazarus
13141314
13151315An understanding of the principles and commands described above is a prerequisite to continuing with this section.
13161316
13171317Unlike Qemu, a container will run at the host computer's full speed. Unlike User Mode Linux (UML), a 32-bit container will run on a 64-bit host. Subject to security issues, this makes Docker the preferred choice when setting up a temporary 32-bit environment to e.g. investigate libraries which would not normally be installed on the host.
13181318
13191319This section discusses installing a 32-bit Debian "Lenny" container, which allows a program to be checked with libraries etc. dating from roughly 2008.
13201320
13211321Referring to <https://github.com/lpenz/docker-debian-releases/>
13221322
13231323 $ docker pull lpenz/debian-lenny-i386
13241324
13251325 $ docker image ls
13261326
13271327 $ docker run --interactive --tty lpenz/debian-lenny-i386
13281328
13291329Lenny authentication is broken and is not going to be fixed (short of forking the entire repository). In /etc/apt/apt.conf put
13301330
13311331 APT
13321332 {
13331333 // Options for apt-get
13341334 Get
13351335 {
13361336 AllowUnauthenticated "true";
13371337 }
13381338 }
13391339
13401340which should allow tasksel etc. to work. Add contrib and non-free to /etc/apt/sources.list, then
13411341
13421342 # apt-get update
13431343
13441344 # apt-get upgrade
13451345
13461346 # tasksel --new-install
13471347
13481348 # dpkg-reconfigure locales
13491349
13501350 # apt-get install openssh-server sudo
13511351
13521352Populate /etc/rc.local as discussed above, comment out the temporary setting from /etc/apt/apt.conf
13531353
13541354Create group and user as discussed above, add user to sudo group.
13551355
13561356Add the build-essential, gdb, subversion and zip packages as basic prequisites. Add libncurses5-dev, libncursesw5-dev and libgpm-dev to allow FPC to be rebuilt, plus possibly libpango1.0-dev for the FP IDE. Install FPC v2.2.4 binary for i386 and corresponding sources, the compiler should run and be able to recompile itself.
13571357
13581358Install the libgtk1.2-dev package then Lazarus 0.9.24.1, don't attempt to run it yet. In the per-user shell startup file e.g. ~/.bashrc put the line
13591359
13601360`export DISPLAY=:0`
13611361
13621362then exit and commit the container as described above and restart with something like
13631363
13641364`$ docker run -it -p 65022:22 -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \`\
13651365`       --entrypoint=/etc/rc.local lenny_with_lazarus`
13661366
13671367Again, details as discussed in the section above.
13681368
13691369It should now be possible to login to the container from the host over SSH, invoke e.g. /usr/local/share/lazarus/lazarus by name, and have the IDE appear on the host computer's screen.
13701370
13711371## Further Reading
13721372
13731373- [Debian on an emulated ARM machine](http://www.aurel32.net/info/debian_arm_qemu.php)
13741374- [Debian 11 on an emulated ARM/ARM64 machine](https://forum.lazarus.freepascal.org/index.php/topic,56098.msg420159.html#msg420159)
13751375- [Debian on an emulated MIPS(EL) machine](http://www.aurel32.net/info/debian_mips_qemu.php)
13761376- [QEMU/Windows XP on Wikibooks](http://en.wikibooks.org/wiki/QEMU/Windows_XP)
13771377- [The User-mode Linux Kernel Home Page](http://user-mode-linux.sourceforge.net/)
13781378- [Installing Debian under Hercules](http://www.josefsipek.net/docs/s390-linux/hercules-s390.html)
13791379
13801380## See also
13811381
13821382- [[Native ARM Systems]]
13831383- [[Native MIPS Systems]]
13841384- [[Small Virtual Machines]]
13851385- [[Docker Containerization]]
13861386
13871387---
13881388*From the Free Pascal wiki, page "Qemu and other emulators", as it stood on 2022-08-16 after a change by Skalogryz, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).*
13891389Category: Docker
13901390Category: Emulators

Back to history