Most of the performance hit comes from KVM.
KVM (Kernel Virtual Machine) is a Linux kernel module that allows a user space program to utilize the hardware virtualization features of various processors: http://wiki.qemu.org/KVM
To enable kvm virtualization support under qemu, you need to pass -enable-kvm option:
$ qemu-system-x86_64 -enable-kvm -m 1024 -hda vm.img
Or you can install qemu-kvm package with apt and run kvm script like that:
$ kvm -m 1024 -hda vm.img