Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Virtualsmp #41

Open
wants to merge 35 commits into
base: master
Choose a base branch
from
Open

Virtualsmp #41

wants to merge 35 commits into from

Conversation

udosteinberg
Copy link
Contributor

The following series of patches adds the necessary synchronization in the VMM to support multiple virtual CPUs on different physical cores. A description of the concepts and mechanisms can be found in Markus' thesis titled "Adding SMP Support to a User-Level VMM"

Jacek Galowicz and others added 30 commits November 15, 2013 12:28
This is needed for:

- luring the VCPU recall context into a blocking semaphore to pause its execution at end of migration.
- unlocking the VCPU before boot to let it run into recall context and restoring it to exact state like on the source migration host.
This is needed for live migration.

The general semantic works like this:
- The caller will ask for some RW-mapped memory range
- This range will be found in the host op
- It will be remapped as read-only and reported back to the caller

This routine uses a pointer which is moved round-robin through the guest memory range.

WARNING: Assumes NOVA as underlying kernel. Was not ported to UNIX.
Added a CpuMessage to add arbitrary offsets to the VCPU's timestamp counter.
This is needed for live migration.
Devices will be attached to this.
The migration code uses this to to communicate with classes of devices.
Devices can write their state into restore messages and also read it back to restore.
…eval.

The live migration module needs this to tell the target host what kind of VMM has to be started.
…which was in use in the Vancouver project on NUL to do host app networking.
This is only compiled into the project and not in use, yet.
The next commit will embedd these mechanisms into main.cc
…d backspace key press.

As this migrates to a hard coded destination host, this could also be done more elegant:
- By a VMCALL from the VM, carrying a magic number in the eax register and the destination host in the ebx register.
- By some VM manager application, triggering the migration event via some IPC event.
- By a fancy ncurses menu, prompting the user for the destination host IP.
ACPI events can be rised with this, fixed and GP events.
The restore procedure does automatically propagate its new position within the LAN.
… methods.

This has been done in the Migration class constructor, but this was too early after reordering VMM parameters for live migration retrieval.
Users of the memory bus can now determine if they are working with actual guest-physmem.
From now on, only actual guest-physmem will be tracked.
It will now only check memory ranges which are actual guest-physmem.
In general the transfer has demonstrated to be errorfree.
However, checksumming is useful to find out if changes on the tracking mechanism etc. provoke data corruption.
The last resend round did tend to be uncomplete in the scenario of both slow ethernet and large Writeable Working Sets.
This commit adds hooks to the DBus infrastructure to install a proxy between message senders and receivers. In addition to the ReceiveFunction, a similar EnqueueFunction is provided which gets called upon message sending. An I/O thread can then register enqueue callbacks for the respective message type and manage sending on the caller's behalf. If the callback returns false or is not present, everything works as before.

Another callback (named "claim") can be used to configure messages to bypass the I/O thread and hence being sent by the issuing thread directly.

All information needed to process the message identically as before is encoded into a new message type MessageIOThread. This information includes the send mode (FIFO, LIFO, early out, round-robin), if the message should be sent synchronously (i.e., the caller has to wait until the request is completed) and which vCPU was the parent of the bus, if applicable.
This commit adds a reference implementation for the unix frontend to show how an I/O thread could be implemented.

Note that the global lock is still in place to allow for easily disabling the I/O thread. Performance may suffer, but the unix frontend is proof-of-concept only, anyway. The I/O thread can be disabled by commenting out the #define USE_IOTHREAD line in unix/main.cc

Access to guest RAM is bypassing the I/O thread because it is synchronized by the operating system.
The vCPU threads are now pinned to consecutive physical cores, starting at the one following the original core that vancouver was started on. As a first simple solution, every physical core gets assigned a dedicated timeout object (i.e., a timer session). Later on, this could be restricted to the actual cores the instance runs on.
This commit ports the reference implementation of the I/O thread found in the unix frontend to NRE. It places the I/O thread worker on the CPU assigned to Vancouver, leaving the vCPUs on the following CPUs.
Markus Partheymueller added 5 commits January 15, 2014 11:12
Sporadic event handlers should have higher priority. In the case of the I/O thread, this is important when it is colocated with another vCPU (which is not advised). For timers, this can help avoid timing issues when the VM does busy waiting on timer events.
Because the synchronization is now provided by the I/O thread, it is safe to remove the global lock.
To help modify vCPU and (LA)PIC subsystem to use atomic operations or fine-grained locking instead of a global lock, this synthetic testing utility can be used to stress the respective device models in an isolated way and run targeted development cycles with it.
Races in the emulation paths of the vCPU and the interrupt controller logic can cause problems when no external synchronization mechanism is applied. Using atomic instructions and relocation of certain code sections, it will now be possible to concurrently access vCPU, Lapic and PIC without the need for a lock around them.
The following devices are now configured to bypass an I/O thread:
* vCPU memory and CpuMessage: Safe as of previous commit.
* PM Timer: No need for synchronization.
* VGA Framebuffer memory: No need for synchronization.
* PCI Pass-through memory and IRQ: No need for synchronization, IRQ already safe.
@blitz
Copy link
Contributor

blitz commented Jan 15, 2014

Thanks!

alex-ab pushed a commit to alex-ab/seoul that referenced this pull request Oct 3, 2023
For further information please look at 'src/app/audio_player/README'.

Fixes TUD-OS#41.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants