Port Linux 2.6 Kernel to Qemu MipsAuthor: Joy Tang2009.1Version 1.02Overview•Use open source to study, no confidential information inside.•If we need deep view inside linux, we must understand what kernel do.•This doc is for build up mips for linux environment run on X86 emulator machine.I, Build Kernel•Target•Main CPU: QEMU emulate Mips•Cross Compile：mips-linux•Develop Environment: Red Hat•Notes, QEMU also support melta board, and MIPSsim. Reference web page: https:///mailarchive/git-commits-head/2008/1/29/636729/thread, from version linux 2.6.25, it remove qemu mips inside.Prepare Working•Prepare Working:•Download source code of Linux:•Annymos login •Go to the folder "/pub/linux/kernel/v2.6" inside FTP.•Get the source code：tar -jxvf linux-2.6.22.19.tar.bz2•2.6.21 will be another choice.Add Toolchain to Search Path•vi /etc/profile, add following lines•PATH="$PATH:/opt/hardhat/devkit/mips/lexra_fp_b e/bin"•PATH="$PATH:/opt/hardhat/devkit/mips/lexra_fp_b e/target/bin"•PATH="$PATH:/opt/build_mips_nofpu/mips-uClibc-linux/usr/bin“•…Change Makefile•vi Makefile, search SUBARCH•Change toSUBARCH := mips•Find CROSS_COMPILE, change toCROSS_COMPILE ?= mips-linux-•vi arch/mips/Makefile, search cflags-y:=, if it is cflags-y:= mips1, change it to cflags-y:=•The change base on user itself environment.Start to Build, Step1•Make config or make menuconfig or make xconfig •Change CPU to QEMU•Add support for “Initial RAM filesystem and RAM disk”. (Initial RAM filesystem and RAM disk (initramfs/initrd) support (BLK_DEV_INITRD) [N/y/?] n change to Y)•Change many other settings, such as MAP_RAM, MTD support, MTD_PLATRAM•Attach .config file for reference. (end of this doc)•File system analyses•In company file system, there are 4 mtds, •1, busybox=> mtd1•2, kernel=>mtd2•3, workdir=>mtd3•4, jffs etc folder=> mtd4•This step will mount all this 4 files.•Change source:go to drivers/mtd/maps.•Run cp cfi_flagadm.c qemuflash.c•Change linux/drivers/mtd/maps/Kconfig, find config MTD_CDB89712, after it, addconfig MTD_QEMU_EMUFLASHtristate"gggggg qemu emu flash”•Also change Makefile under linux/drivers/mtd/maps/ Find obj-$(CONFIG_MTD_CDB89712)+= cdb89712.o, after it, addobj-$(CONFIG_MTD_QEMU_EMUFLASH)+= qemuflash.o•See attach (makefile& kconfig, end of this doc)•For the file, qemuflash.c, there is some comments:•mtd0,mtd1, mtd2 is not used. Mtd3 is for workdir, mtd4 is for jffs etc folder.•Here is attach. (end of this doc)•Go to folder \fs\jffs2, open earse.c, line 350, function jffs2_block_check_erase, add a qemu patch#ifdef CONFIG_QEMU#elseprintk(KERN_WARNING"Newly-erased block ……#endif•Goto root folder, run make config, press enter until it quit. This is for correct errors in the file “.config”•Run build commandmake all•Run commandcp -f ./vmlinux/opt/emularor/qemu/mips/myimages2…After that, kernel elf file will be send to qemu folderII, Build Qemu•Download qemu packet on web/qemu/download.html•Get the file /qemu/qemu-0.9.1.tar.gz •Uncompress it. In order to save time, inside file config-host.mak: change TARGET_DIRS to mips-softmmu.•Open root folder\vl.c•Add following lineschar g_mtdFileName_etcSave[1024];char g_mtdFileName_workdir[1024];•At beginning of main function, addmemset(g_mtdFileName_etcSave, 0, 1024);memset(g_mtdFileName_workdir, 0, 1024);At the end of function drive_init, add following lines if (type == IF_PFLASH && g_mtdFileName_etcSave[0] == 0){//mtd will load to bios mem, save file name firstsprintf(g_mtdFileName_etcSave, "%s", file);}if (type == IF_MTD && g_mtdFileName_workdir[0] == 0){//mtd will load to bios mem, save file name firstsprintf(g_mtdFileName_workdir, "%s", file);}•Go to “hw”folder, open mips_r4k.c, add following lines •extern char g_mtdFileName_etcSave[1024]; //2M •extern char g_mtdFileName_workdir[1024]; //30M•#define START_ADDR_ETC_SAVE 0x1fc00000•#define SIZE_ETCSAVE 0x00200000•#define START_ADDR_WORKDIR 0x1fe00000•At the end of function, mips_r4k_init, add following lines, add processing of g_mtdFileName_etcSave andg_mtdFileName_workdir.Build QEMU Step 3(continue) if (g_mtdFileName_etcSave[0]){bios_offset= ram_size+ vga_ram_size;bios_size= load_image(g_mtdFileName_etcSave, phys_ram_base+ bios_offset);if ((bios_size> 0) && (bios_size<= BIOS_SIZE))cpu_register_physical_memory(START_ADDR_ETC_SAVE,BIOS_SIZE, bios_offset| IO_MEM_ROM);else/* not fatal */ fprintf(stderr, "qemu: Warning, could not load MIPS bios '%s'\n",buf);}if (g_mtdFileName_workdir[0]){bios_offset= ram_size+ vga_ram_size+ SIZE_ETCSAVE;bios_size= load_image(g_mtdFileName_workdir, phys_ram_base+ bios_offset);if ((bios_size> 0) && (bios_size<= BIOS_SIZE))cpu_register_physical_memory(START_ADDR_WORKDIR,BIOS_SIZE, bios_offset| IO_MEM_ROM);else/* not fatal */fprintf(stderr, "qemu: Warning, could not load MIPS bios '%s'\n",buf);}Attach mips_r4k.c, end of this doc•Open root folder sysemu.h, change 4m to 32m#define MAX_BIOS_SIZE (32 * 1024 * 1024)…#elif defined(TARGET_MIPS)#define BIOS_SIZE (32 * 1024 * 1024) //we want use bios ram to be mtd area #endif•Here is attach file for it. (end of this doc)•Run following command: ./configuremakemake installIII, Build busybox •Find web /downloads/, choose/downloads/busybox-1.6.1.tar.bz2•Uncompress it.•Change make file, like the build kernel,•vi Makefile, search SUBARCH•Change toSUBARCH := mips•Find CROSS_COMPILE, change toCROSS_COMPILE ?= mips-linux-Build busybox Step2•make xconfig•Change settings for it, make sure the mount & fdisk is inside.•make\n•make install\n•cd./_install\n•rm dev -rf\n•cp /opt/source/flashmaster_prxxx_fe-1.0.linux-2.6-2/res/1.0/rootfs.mips_nofpu/dev/ ./dev -rf\n•cd-\n•cp -f --reply=yes /opt/source/flashmaster_prxxx_fe-1.0.linux-2.6-2/res/1.0/rootfs.mips_nofpu/sbin/portmap ./_install/sbin\n•../flashmaster_prxxx_fe-1.0.linux-2.6-2/bin/mkcramfs -Nb_install/ a.mtd\n •cp -f --reply=yes a.mtd../emulator/qemu/mips/\nIV,MTD files, step 1•There are 4 MTD files in real system, now we design each to a file,•A, MTD1, it will be busy-box => -initrd“filename”•B, MTD2, it will be kernel=>-kernel “filename”•C, MTD3, it will be workdir=>-mtdblock“filename”•D, MTD4, it will jffs2 format etc/save=>-pflash“filename”•MTD files Step 2•For 1st, busybox, run following command to create it cd/opt/…/flashmaster_prxxx_fe-1.0.linux-2.6-2/res/1.0 mkdir rootfs.mips_nofpu/dev/netmknod rootfs.mips_nofpu/dev/net /tun c 10 200chmod0666 rootfs.mips_nofpu/dev/net /tun../../bin/mkcramfs -Nb rootfs.mips_nofpu a.mtd•or directly use exist one “mtd1.img”.•For 2nd, use build kernel file “vmlinux”•For 3rd, use workdir•For 4th, run following command../../bin/mkfs.jffs2 -o etcsave.mtd-d ./etc -e 65535 -p ff -bComments of MTD•For 1st, parameter –initrd use it as ram-disk. It will loaded to ram of qemu memory space.•For 2nd, parameter –kernel use same concept of 1st.•For 3rd, it emulate a flash, it start address will be 2M, 0x1fc00000 + 0x00200000, size 30M, 0x01E00000•For 4th, its start address will be 0M0x1fc00000 + 0x0000000, size 2MV, Create script fileto run kernel •Its contentsinitarg="init=/linuxrc root=/dev/hda root2=/dev/hd18" mtdparameter="-mtdblock./boot/workdir.mtd“pflashparameter="-pflash./boot/etcsave.mtd“time \${qemupath}qemu-system-mips\-kernel ./myimages2/vmlinux \-initrd a.mtd\-append "console=ttyS0 $initarg" \-nographic\-m 128 \$pflashparameter\$mtdparameter\-net nic,vlan=1 -net user,vlan=1Comments for the script file•-nographic can be removed. If it is exist, qemu emulator will not show command console, we can use secure-crt’s SSH2 account to do this.•If it is not exist, we must run under x window in linuxAdd function for readphysical memory in QEMU •Add a function in monitor.c, do_phymemory_save •Detail is in attachment monitor.cFor run qemu command•Use graphic, ctrl+alt+2•Run command•Here is help:•help|? [cmd] --show the help•commit device|all--commit changes to the disk images (if -snapshot is used) or backing files •info subcommand --show various information about the system state•q|quit--quit the emulator•eject [-f] device --eject a removable medium (use -f to force it)•change device filename --change a removable medium•screendump filename --save screen into PPM image 'filename'•logfile filename --output logs to 'filename'•log item1[,...] --activate logging of the specified items to '/tmp/qemu.log'•savevm tag|id--save a VM snapshot. If no tag or id are provided, a new snapshot is created •loadvm tag|id--restore a VM snapshot from its tag or id•delvm tag|id--delete a VM snapshot from its tag or id•stop --stop emulation•c|cont--resume emulation•gdbserver[port] --start gdbserver session (default port=1234)•x /fmt addr--virtual memory dump starting at 'addr'•xp/fmt addr--physical memory dump starting at 'addr'•p|print/fmt expr--print expression value (use $reg for CPU register access)•i /fmt addr--I/O port read•sendkey keys --send keys to the VM (e.g. 'sendkey ctrl-alt-f1')•system_reset--reset the system•system_powerdown--send system power down event•sum addr size --compute the checksum of a memory region•usb_add device --add USB device (e.g. 'host:bus.addr' or 'host:vendor_id:product_id')•usb_del device --remove USB device 'bus.addr'•cpu index --set the default CPU•mouse_move dx dy[dz] --send mouse move events•mouse_button state --change mouse button state (1=L, 2=M, 4=R)•mouse_set index --set which mouse device receives events •wavcapture path [frequency bits channels] --capture audio to a wave file (default frequency=44100 bits=16 channels=2)•stopcapture capture index --stop capture•memsave addr size file --save to disk virtual memory dump starting at 'addr' of size 'size'•pmemsave addr size file --save to physical memory memory dump starting at 'addr' of size 'size'VI,Study on mount cd iso,harddiskand share files•1)For mount hda and cdrom correctly, we need open following kernel options:•A, ATA IDE support option.•B, VFS support•C, CDROM ISO9660 file system support•(“make xconfig”to find&open it. Recompile and build kernel again)•2)Go to /dev folder inside root file system prepare folder, run following command:•mknod hdc b 22 0•3)inside run-qemu script file, add•-cdrom./cd.ISO-hda linux.img.•(cd.ISO can be any cdrom iso file, linux.img copy fromhttp://lassauge.free.fr/qemu/release/Qemu-0.9.1-windows.zip. ./share is a folder for share between qemu and host x86 linux system)•4)After all file prepared, in qemu kernel, test this:•mount /dev/hda/tmp3 => hda will be linux.img’s data•mount /dev/hdc/tmp4 => hdc will be cdrom’s dataMount(continued)Now I want to share data with vmware. Here is steps to share data with vmware:1, inside vmware, create a IDE harddisk disk, select persistent and independent. Minimum size is 0.1GB, (100MB). Suppose it use hdd inside vmware, its file name is “vmware-qemu-flat.vmdk”. When generate it, choose a special folder for it. It is for share purpose.2, let the share folder inside a folder /mnt/hgfs. (The other topic, vmware tools.)3, inside vmware, run command: fdisk/dev/hdd. Use “n”command to create a new partition.4, inside vmware, run mkfs/dev/hdd. Then vmdk file is formatted. Copy new vmdk file to qemu’s folder, suppose it is “./share”. Make sure don’t format other partition inside vmware. ☺5, add parameter in run qemu-mips-system script: -hda/mnt/hgfs/share-qemu/vmware-qemu-flat.vmdk. Start the qemu-mips-system. Use this command in vm: mount /dev/hda/tmp3.6, With upper steps, the vmdk file can be share between qemu and vmware. It will be useful to share files.7, Before transfer the vmdk files between vmware and qemu, pls run “sync”. It is for write buffer’s data. “sync”may not enough. Maybe need “poweroff”VII. GDB for debug kernel•Step1, need re-build kernel. Run “make xconfig”, open many options in “kernel hacking”, such as “compile kernel with debug info”etc.After compile, we can find kernel image size change from about 4m to about 20m. This is normal.•Step2,use X window linux, in command script, remove –nographic, add a paramter–S, it is for stop kernel and wait for gdb command.•Step3, after qemu window start, you will see black screen but press Ctrl-Alt-3 to see bash prompt. Press Ctrl-Alt-2, go to qemu command window, press “gdbserver”, at this time, qemu will start to listen port 1234.•Step4, start another linux terminal, run “mips-linux-gdb vmlinux”, in gdb, run command “(gdb)target remote localhost:1234”, you will find gdb can trace kernel now.VIII network1, host preparation in linux •From /progDownload/tunctl-Download-41592.html download tunctl-1.5.tar.gz•Comment 2 lines,•#docbook2man $(PACKAGE).sgml•#install $(MAN) $(DESTDIR)$(MAN_DIR)•make•make install•Find if host linux kernel have “tun.ko”•[root@...~]# find /lib -name "tun.ko"•/lib/modules/2.6.9-22.EL/kernel/drivers/net/tun.konetwork•Goto guest kernel folder, run “make xconfig”•Find “network device support”=> “other ISA cards”=>”ne2000/ne1000 support”, open it.•Re-compile your guest kernel after the change.network•-----------------------------------------"create /usr/local/sbin/tap.sh file (e1000g0 is network card label, this file is at host) -----------------------------------------#!/bin/sh•IF=e1000g0•TAP=tap0•case "$1" in•"start")•sudo tunctl-t $TAP•ifconfig$TAP plumb•brdgadm-a $IF•brdgadm-a $TAP•;;•"stop")•#brdgadm-d $IF•#brdgadm-d $TAP•ifconfig$TAP unplumb•sudo tunctl-d $TAP•;;•esac•exit 0•------------------------------------# /usr/local/sbin/tap.sh start## /usr/local/sbin/tap.sh stop------------------------------------network•After run “tap.sh start”on host of linux, we check if tun.ko is probed: “lsmod | grep tun“,•tun 9153 0•And also we check if tap0 device is power up,•ifconfig –a, we can find a new device tap0 is power on.•We can run following command in host linux:“ifconfig tap0 192.168.200.1“.•Create a file, vi /etc/qemu-ifup#!/bin/shifconfig$1 192.168.200.10Network2, guest•Add following string to qemu-run script:•-net nic,vlan=1,macaddr=00:aa:00:00:01:01 -net tap,vlan=1,ifname=tap0•Startup qemu-system-mips by the added script file.•In guest linux shell, run:•ifconfig eth0 192.168.200.11•ping -c 4 192.168.200.1 to check if network is ok.Network error•I meet this error:•# ping -c 3 192.168.200.10•PING 192.168.200.10 (192.168.200.10): 56 data bytes•NETDEV WATCHDOG: eth0: transmit timed out•NETDEV WATCHDOG: eth0: transmit timed out•---192.168.200.10 ping statistics ---• 4 packets transmitted, 0 packets received, 100% packet loss•After I change this file, arch/mips/kernel/i8259.c , I solve the issue on guest linux2.6.21.•add .disable = disable_8259A_irq,on line 39•[root@localhost linux-2.6.21.7]#diff ./arch/mips/kernel/i8259.c ./arch/mips/kernel/i8259_test.c•39d38•< .disable = disable_8259A_irq•Reference web for the patch:•/mm-commits@/msg24505.html or •/Mailing-Lists/Kernel/2007-07/msg12733.html3, Network prepare inhost Windows XP host system•1, go to /BBS/thread-3209-1-1.html, download tapnic.rar.•After click “addtap.bat”, a new driver for virtual XP network is generated. Please rename it to tap0.•Reference web http://www.h7.dion.ne.jp/~qemu-win/TapWin32-en.html.XP windows hostprepare for network •Force IP address of tap0:Windows XPprepare•Run this command for windows XP:•qemu-system-mips.exe-L . -m 192 -kernel .\myimages2\vmlinux -initrd a.mtd-append "console=ttyS0 init=/linuxrc root=/dev/hda pci=noapicisa=noapic" -M mips-cdrom./cd.ISO-hda./share/vmware-qemu-flat.vmdk-net nic,vlan=1,macaddr=00:aa:00:00:01:01 -net tap,vlan=1,ifname=tap0,vlan=1,ifname=tap0•Guest OS preparation same as upper mentioned hostlinux+guest linux.Rootfs changefor the network•In order to mount nfs server on host linux or windows host, I add nfs support in guest kernel option, but still cannot mount nfs folder.•Later I add follow lines in rc.sysinit and solve the unable mount issue.ifconfig lo 127.0.0.1netmask255.0.0.0 broadcast 127.255.255.255 portmapmount -t ramfs ramfs/tmp-o maxsize=4000More Host Preparationfor Network•1, windows, for NFS work, need install cygwin, make sure NFS module is installed. We need edit “C:\cygwin\etc\hosts”and “C:\cygwin\etc\exports”for NFS setting•2, Linux, make sure NFS work. It is at “start menu”=>”system setting”=> “server setting”=> “nfs setting”Malta board(Mips24k)& Graphics study•MTD_PHYSMAP need to be closed.•Because Malta board has graphics card support, now I start to study malta board.•1) get svn code from internet:•svn co svn:///qemu/trunk•2)don’t change anything on qemu code like before because there is a crash on change code and I find workdir also can be mounted on /dev/hdc. Also don’t need to change guest kernel code, it can be optional.•3)copy malta/.config to guest kernel folder. Build guest kernel with 2 commands: make config, make allMalta (continue)•4, create a script file, my-run-qemu without graphic support(in attachment, /malta/script) and another script file, graphics-qemu with graphic support. •run the graphic script file. For malta board, we don’t need change arch/mips/kernel/i8259.c like the fake qemu board.•Under guest linux, run “mount /dev/hdc/tmp3”to check if workdir is mounted on /tmp3.Some capture imageon malta platform•Graphic display after press “ctrl+alt+1”•After run some command, network mount is working.Futher topic & summerize•There are still 1 important topics Emulate every device inside Mips chip.•That’s all•Because pdf file don’t support *.rar file asas attachment, pls rename the txt to rar.。
