Allow guest and (KVM hypervisor) host to communicate when using macvtap
I’ve been trying the version of Virtual Machine Manager that comes with Fedora 16, and I have to say that it’s quite cool.
On the network front, it’s easy to configure a bridge with the host’s network card using macvtap, so the guest works as if it had a real Ethernet card plugged on the LAN. But I quickly discovered that the host and the guest were not able to ping each other!
It took me a lot of time to find the (not so easy) solution on the Web, so here it is.
Quick overview
The solution is to configure a macvlan interface on the hypervisor, with the same IP address than the real hardware interface, and to configure routing to use it. In Qemu/KVM, use a macvtap interface on the hardware interface as usual.
Instructions (on the Hypervisor host)
We have to use the “ip” tool (from the iproute2 package), so the one like me who were used to ifconfig will have some new stuffs to learn. In the following commands, p10p1 is my network card (might be eth0 for you). My network uses the 192.168.1.0/24 IP range, and the gateway is 192.168.1.1.
First, add a macvlan interface called “macvlan0″ on top of the hardware interface p10p1 (in bridge mode). Assign the same IP address than the one for p10p1 (This is the magical trick that makes it work).
ip link add link p10p1 address 00:19:d1:29:d2:58 macvlan0 type macvlan mode bridge ip address add 192.168.1.100/24 dev macvlan0 ip link set dev macvlan0 up
Then, remove all routing rules related to the real hardware interface, and configure a default route using the new macvlan0 interface:
ip route flush dev p10p1 ip route add default via 192.168.1.1 dev macvlan0 proto static
And that’s it!
You should put these in a script that is ran when the hardware interface, but until I have some time to waste I have no idea how to do it with NetworkManager!
December 14, 2011
· 
npen · 
4 Comments
Tags: Fedora · Posted in: Tutorial
Yet another simple example of Comet in Java using Glassfish, Grizzly, JSONP and JQuery
I wanted to try long poll queries (aka Comet), and also fully understand how it works, but the simple examples I found weren’t perfect so here is mine!
You can try it there (use two different browser windows, or two different browser): http://www.lostgeeks.org:8080/comet-test/ (Yes I run Glassfish on a virtual server with 512MB of RAM, Apache, Exim, MySQL, Dovecot…!)
You can download the complete Maven project here : comet-test.zip
I’m using the following technologies:
- Java 7
- Glassfish 3.1.1 (Java EE 6)
- Grizzly (the HTTP framework used by Glassfish. It uses Java NIO’s non-blocking IO, so long poll request don’t require one thread per client)
- JQuery
- Maven
- Netbeans 7.0.1
Overview
This the simple global counter example. All the users on the page:
- Can see the current value of the counter
- Can increment the counter
- Gets immediately notified when someone in the world increments the counter (that’s the Comet part)
We will use a POST request to a Servlet to increment the counter, and a GET request to the same Servlet to get its value. To avoid some limitations in the browsers (connection limit to the same domain), we’ll use two different domains (subdomains is also OK). This forces us to use a technique called “JSONP” to bypass the restriction of XmlHttpRequest (aka XKR) accross different domains: a <script> tag with a source pointing to the different domain is generated in the DOM, and the URL simply returns JSON encapsulated in a javascript method call. Thanks to JQuery for hiding all this ugliness!
Enable Comet support in Glassfish
This is an important step you shouldn’t forget: You have to enable Comet support in Glassfish for your HTTP listener. If you ask Google, it will tell you how. But if you run Glassfish via Netbeans, you need to check a box in the Tools → Servers menu.
Create a servlet for processing the request
I prefer to use a servlet for such simple example, as it’s easier to follow what happens. I use servlet 3.0 annotations so there is no need for web.xml configuration.
import java.util.concurrent.atomic.AtomicInteger;
@WebServlet(name = "CometServlet", urlPatterns = {"/CometServlet"}, loadOnStartup=1)
public class CometServlet extends HttpServlet {
private final String TOPIC = "CounterCometService";
private final AtomicInteger counter = new AtomicInteger();
...
}
As you can see, the servlet has two final members (which means they are threadsafe). The TOPIC string will be used as a key for the Comet context. Counter will be used to maintain a “global” state. It will be accessed by different threads, that’s why we use an AtomicInteger object.
The Javascript code
The Javascript code is directly present in the index.xhtml (JSF) page. This is useful for generating the URL using EL (Expression Language). The polling URL is generated using an application scoped bean, that gets a custom resource injected (That is to say the secondary URL for polling is parametrized in a custom String resource in Glassfish).
Notice the callback=? parameter in the polling URL. It indicates JQuery that it should use JSONP. JQuery will replace the question mark with a generated Javascript function name, and this function name will be used server side to encapsulate the JSON data.
var postUrl = "#{application.contextPath}/CometServlet";
var pollUrl = "#{poll.domain}" + postUrl + "?callback=?";
function updateCount(c, s) {
$('#count').html(c);
$('#source').html(s);
}
function poll(immediate) {
var url = pollUrl;
if (immediate) url = url + "&immediate=yes";
$.getJSON(
url,
function(data) {
updateCount(data.Counter, data.Source);
poll();
}
);
}
function increment() {
$.ajax({
type: 'POST',
url: postUrl
});
}
function init() {
poll(true);
$('#postUrl').html(postUrl);
$('#pollUrl').html(pollUrl);
}
$(window).load(init);
I think the increment function is self explanatory. The poll function takes a boolean parameter “immediate”, which is passed to the server if its value is true. It’s just a trick to make the servlet return the counter value immediately, without waiting for an event. It is used on page load to display the initial value of the counter. You can also see that the servlet returns an object with two fields : Counter is the counter value, and Source is just a string to show from what kind of event the counter value comes from :
- “IMMEDIATE” indicates that the result comes from a GET request with the immediate param set to yes.
- “EVENT” indicates that the result comes from an increment event.
- “INTERRUPT” indicates that the polling request reach the timeout set on server side.
In all cases, the display is updated, and the poll request is run again.
Server side code
Initialize the servlet
In the servlet init() method, we register our topic to the comet engine, and set a delay of 20 seconds before request expiration.
import com.sun.grizzly.comet.CometContext;
import com.sun.grizzly.comet.CometEngine;
@Override
public void init(ServletConfig config) throws ServletException {
super.init(config);
CometEngine engine = CometEngine.getEngine();
CometContext cometContext = engine.register(TOPIC);
cometContext.setExpirationDelay(20 * 1000);
}
Writing the response to the GET request
We define a method in the servlet to write the response (Note that this method is threadsafe). The jsonpCallback string contains the name of the Javascript callback function, it is passed via the callback parameter of the request. The source parameter is a tag passed to the client to indicate the source of the response.
private void writeJsonp(HttpServletResponse response, String jsonpCallback,
String source) throws IOException {
int count = counter.get();
response.setContentType("text/javascript");
response.getWriter().write(jsonpCallback +
"({'Counter': " + count + ", 'Source': '" + source + "'})");
}
Implementation of the GET method for immediate request is as follow:
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String callback = request.getParameter("callback");
String immediate = request.getParameter("immediate");
if ("yes".equals(immediate)) {
writeJsonp(response, callback, "IMMEDIATE");
} else {
...
}
The CometHandler implementation
We need an object that implements the CometHandler interface. It will receive the events from the CometEngine. The class is defined as a inner class of the servlet, so it will have access to our writeJsonp() method.
private class CounterCometHandler implements CometHandler {
public CounterCometHandler(String jsonpCallback) {
this.jsonpCallback = jsonpCallback;
}
private final String jsonpCallback;
private HttpServletResponse response;
@Override
public void attach(HttpServletResponse attachment) {
this.response = attachment;
}
@Override
public void onEvent(CometEvent event) throws IOException {
if (CometEvent.NOTIFY == event.getType()) {
writeJsonp(response, jsonpCallback, "EVENT");
event.getCometContext().resumeCometHandler(this);
}
}
@Override
public void onInitialize(CometEvent event) throws IOException {
}
@Override
public void onInterrupt(CometEvent event) throws IOException {
writeJsonp(response, jsonpCallback, "INTERRUPT");
event.getCometContext().resumeCometHandler(this);
}
@Override
public void onTerminate(CometEvent event) throws IOException {
remove(event);
}
private void remove(CometEvent event) throws IOException {
Closeables.closeQuietly(response.getWriter());
CometContext context = event.getCometContext();
context.removeCometHandler(this);
}
}
Some explanations:
- We pass the jsonp callback in the constructor to save it for later use.
- The HttpServletResponse is saved in the attach callback (The CometHandler is parametrized with this type).
- The OnEvent() method is where we get notification of change of the counter, so we write the new value, and call resumeCometHandler to finish the request. Note we can get the CometContext directly from the event.
- The OnInterrupt() method is called when the timeout is reached. It might also be called if the client close the connection however, so this implementation might not be 100% correct.
- The OnTerminate() method seems to never be called for me, so I cannot tell what it is for (and the documenation is not very clear). The implemenation is taken from other examples on the web.
The Comet part of the doGet method
In this part of the doGet method (when no immediate=yes parameter is given), we create a new handler, and add it to the CometContext. This has the effect to suspend the connection until something happens.
CounterCometHandler handler = new CounterCometHandler(callback); handler.attach(response); CometEngine engine = CometEngine.getEngine(); CometContext context = engine.getCometContext(TOPIC); context.addCometHandler(handler);
Incrementing the counter (And notifying all the waitings handlers)
We use a POST request (without any parameters) on the servlet to increment the counter.
Finaly, we use notify(Object o) on the CometContext to notify all the waiting handlers installed in the doGet() method. (Note that the notify() method without any parameter is the standard Object method, and should not be used here!). The passed string is not used anywhere, and it could be replaced with null.
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
counter.incrementAndGet();
CometEngine engine = CometEngine.getEngine();
CometContext context = engine.getCometContext(TOPIC);
context.notify("Ola !");
}
That’s all !
This has been tested on Firefox 6, Chromium 13 and even on my old Android phone (HTC Magic)!
August 20, 2011
· npen · 5 Comments
Tags: Java · Posted in: Tutorial
Squeeze
This (virtual) server has been updated to Debian 6 (Squeeze) ! It started with Debian 4 (Etch) and had already been updated to Debian 5 (Lenny).
~# cat /etc/debian_version 6.0.1
It seems everything is still working fine. 
May 15, 2011
· npen · Comments Closed
Tags: Debian · Posted in: Uncategorized
Some news
I’s been a long long time since I’m interested in the graphic stack of Linux (I started using Linux with XFree86 3.3.3 I think, in 1999), and I’m still happy to play with latest Xorg and Mesa after hours of not-so-interesting paid job. I made two small contributions recently, and I’m quite happy they were both accepted:
- The first one is a fix for a build failure I encountered on Xorg’s Xserver while playing with Systemtap. While the patch is quite small, I had to dive into quite complex Automake files to understand what was going on, so I’m quite proud of it (The commit on freedesktop.org’s git repository)
- The second one is on the r300g gallium driver of Mesa (It’s a software driver for some ATI Radeon graphic cards). It fixes some rendering artifacts in some conditions, and the bug was really difficult to spot. (The commit on freedesktop.org’s git repository)
I also made some code review for some simple patches (for memory leaks) on the Xorg Devel mailing list, I found it’s quite a good way to look at new code.
March 28, 2011
· npen · Comments Closed
Tags: Mesa, Xorg · Posted in: Uncategorized
Welcome Arch Linux!
I switched from Fedora to Arch Linux some weeks ago (After a bad experience with Rawhide… my fault), and I’m very happy with it:
- It took me some time to configure everything I need, but I have to say that the documentation (on the wiki) is very well done, and up to date;
- There’s nothing to hide “complexity”, so it’s easy to understand how the system works;
- The concept of rolling release is great;
- It was really easy to build and use (without breaking the system) an up to date X+mesa stack from git.
Of course I’m still using Debian on this (virtual) server.
March 28, 2011
· npen · Comments Closed
Tags: Arch Linux · Posted in: Uncategorized
r300g now the default in Fedora 14 (beta)
Now that I switched (some time ago) from Debian to Fedora on my main computer, I’m glad to see that the gallium driver for R300-R500 cards is now the default in Fedora 14 (in beta now). And I’m also glad to see my name in the Mesa package Changelog 
* Thu Sep 23 2010 Dave Airlie <airlied@redhat.com> 7.9-0.8 - fixup selecting r300 gallium driver (thanks to Nicolas PENINGUY)
/me wants to work on free software !
October 10, 2010
· npen · Comments Closed
Tags: Fedora, Xorg · Posted in: Uncategorized
Parallel build of Debian packages
Hmm, I really need to remember this one! To build Debian packages faster on multi-core CPU, you can use the following:
export DEB_BUILD_OPTIONS='parallel=4' fakeroot debian/rules binary
March 20, 2009
· npen · Comments Closed
Tags: Debian · Posted in: Uncategorized
Testing the r6xx-r7xx-support branch on Debian
It’s been a while since I last wrote on this blog! I’m now a happy Radeon HD 4850 video card owner, so I’d like to try the latest r6xx-r7xx-support branch of mesa, drm, and xserver-xorg-video-ati.
I’m running Debian GNU/Linux Sid (unstable), with some packages (Xorg) from experimental. The goal is to build clean Debian packages to test the new R600/R700 stuff. Thank to Git and The X Strike Force, this should be quite easy!
The DRM part
The DRM part provides both user land library, and kernel modules. On Debian, the libdrm package provides only the library, and the DRM kernel modules comes from the linux-image package (I’m running kernel 2.6.26 right now, so it’s pretty outdated). So the solution is the drm-snapshot source package : once build, you will get a drm-modules-source package, you will have to build against your current running kernel.
First, get the drm-snapshot source tree from the Debian git repository :
git clone http://git.debian.org/git/pkg-xorg/lib/drm-snapshot.git cd drm-snapshot
Let’s see available branches :
$ git branch -r origin/HEAD origin/debian-experimental origin/master origin/upstream-experimental
The debian-experimental branch is what we want, so let’s create a local branch tracking it:
git checkout -t origin/debian-experimental
But we also want the code in the main DRM repository, so we need to add it as the “upstream” remote (tracked) repository. The git fetch command is here to actually get the data.
git remote add upstream git://git.freedesktop.org/git/mesa/drm git fetch upstream
This will give you a lot of branches, that you can list with git branch -r. The branch we want is upstream/r6xx-r7xx-support:
git checkout -t upstream/r6xx-r7xx-support
Let’s see the local branches :
$ git branch master * r6xx-r7xx-support debian-experimental
Come back on the debian-experimental branch, and merge the r6xx-r7xx-support:
git checkout debian-experimental git pull . r6xx-r7xx-support
Here it is! We’re almost ready to build our package! Let’s create a custom version for our package:
git tag -a drm-snapshot-2.4.4+git+20090221+993d387-1 -m "Test build."
20090221 is the date, 993d387 is the revision of the last upstream commit (git log will give it in the merge message), and 1 is the package version with this snapshot.
Then, create a .git-revision file to make the build process happy:
git describe --abbrev=17 > .git-revision
And update the debian/changelog file:
drm-snapshot (2.4.4+git+20090221+993d387-1) experimental; urgency=low * r6xx-r7xx-support branch test build. -- Nicolas PENINGUY <xxxx@xxxxgeeks.org> Sun, 22 Feb 2009 00:12:00 +0100
I use the LANG=C date -R command to get the date, not sure if this is correct.
Finally, build the packages:
fakeroot debian/rules binary
And install them:
cd .. dpkg -i *.deb
Then compile and install the drm modules for your running kernel:
module-assistant a-i drm-modules
The xserver-xorg-video-ati part
This part is quite similar :
git clone http://git.debian.org/git/pkg-xorg/driver/xserver-xorg-video-ati.git cd xserver-xorg-video-ati git checkout -t origin/debian-experimental git remote add upstream git://git.freedesktop.org/git/xorg/driver/xf86-video-ati git fetch upstream git checkout -t upstream/r6xx-r7xx-support git checkout debian-experimental git pull . r6xx-r7xx-support
Update the debian/changelog file :
xserver-xorg-video-ati (1:6.11.0-2) experimental; urgency=low * r6xx-r7xx-support branch test build. -- Nicolas PENINGUY <xxxx@xxxxgeeks.org> Sun, 22 Feb 2009 00:12:00 +0100
And build the package :
fakeroot debian/rules binary
Update: the r6xx-r7xx-support branch has been merged few days ago!
February 22, 2009
· npen · Comments Closed
Tags: Debian, Xorg · Posted in: Uncategorized
Using oprofile with Xorg
It took me some time to understand how to use oprofile efficiently. Oprofile is a system wide profiler, useful to understand why your system is so slowww.
I’ve used oprofile to understand why my system was so unusable when using EXA accelaration with Xorg and a R300 Radeon card. Today things have improved a lot, but I’d like to keep a record on how to use it.
First, it’s useful to have a non-compressed kernel image, so I build my own linux kernel.
Secondly, I followed instructions at http://wiki.x.org/wiki/Development/git to rebuild the current developpement version of the XServer. This gives better results than using stripped binaries of your distribution.
Here is the script I use. The first parameter is the destination directory for the log files.
#!/bin/sh KernelBuildDir=/lib/modules/`uname -r`/build Outdir=$1 sudo opcontrol -h sudo opcontrol --init sudo opcontrol --reset sudo opcontrol --setup --separate=none -e CPU_CLK_UNHALTED:100000:0:1:1 --vmlinux=$KernelBuildDir/vmlinux -c 0 sudo opcontrol --start cat >/dev/null sudo opcontrol --stop sudo opcontrol --dump mkdir -p $Outdir sudo opreport --long-filenames -p $KernelBuildDir -x -t 1 > $Outdir/system.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols /lib/i686/cmov/libc-2.7.so > $Outdir/libc.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols /usr/local/xorg/lib/xorg/modules/libexa.so > $Outdir/libexa.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols /usr/local/xorg/lib/libpixman-1.so.0.11.1 > $Outdir/libpixman.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols /usr/local/xorg/lib/xorg/modules/drivers/radeon_drv.so > $Outdir/radeon_drv.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols /usr/local/xorg/bin/Xorg > $Outdir/Xorg.oprofile sudo opreport -p $KernelBuildDir -t 1 --symbols $KernelBuildDir/vmlinux > $Outdir/vmlinux.oprofile echo "Profile reports saved in $Outdir"
When you start this script, it will profile the system until you press Ctrl-D (That’s what the cat > /dev/null line is for). The first opreport command generate a general view on where the time was spend. Next lines generate specific reports about each library/program.
With this script, time spent in libraries and programs are separated. You might want to use the --separate=kernel flag on set up to get per-process timings. With this option, -c flag is also useful to get a backtrace of time consuming functions.
May 26, 2008
· npen · Comments Closed
Posted in: Uncategorized
About x11perf -aa10text relevance
There have been a lot of interesting blog articles lately about text rendering performance on Xorg with EXA acceleration (wrt XAA)… They always mention the use of the standard x11perf tool (with options -aa10text or -aa24text) to measure antialias text rendering performance. But the results I had were always very different from what was expected !
First, the result with XAA (3 280 000.0/sec) was much better than with EXA (82 200.0/sec). Moreover, the recent implementation of hardware acceleration for RENDER extension on R300 didn’t seem to improve things at all !
After quite some time thinking about it, I realized that the text didn’t seemed antialiased at all !
Looking at the x11perf source code, you can see it asks for this font :
"charter:antialias=true:rgba=0:pixelsize=10"
You can easily find which fonts it gets using fc-match :
fc-match --verbose "charter:antialias=true:rgba=0:pixelsize=10" ... antialias: FcFalse(s) ... file: "/usr/share/fonts/X11/75dpi/charR10.pcf.gz"(s)
Uh ?! The font is not antialiased ! But… PCF fonts are bitmaps fonts, and I don’t think they can ben antialiased anyway !!
I was able to resolve this issue by using a custom fontconfig configuration file in my home directory (the file name is .fonts.conf). It replaces the Charter family with the generic Serif family. That’s a truetype font on my system (probably Bitstream Vera).
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<match target="pattern">
<test qual="any" name="family"><string>Charter</string></test>
<edit name="family" mode="assign"><string>Serif</string></edit>
</match>
</fontconfig>
So now I should be able to do relevant tests to compare EXA and XAA performance on R300.
April 29, 2008
· npen · Comments Closed
Tags: performance, Xorg · Posted in: Uncategorized