[Home]FixingSMIIssues

LinuxCNCKnowledgeBase | RecentChanges | PageIndex | Preferences | LinuxCNC.org

Fixing Realtime problems caused by SMI on Ubuntu

1. Intro
2. Turning off SMI may be dangerous
3. Typical SMI interrupt
4. Configuring SMI on Hardy or Lucid (Ubuntu 8.04 or 10.04, any LiveCD installation newer than about 2008 (emc2-2.3.0))
5. configuring SMI.ko on Dapper or any other installation where it is not present by default
6. Changing smi-module.c
7. Compiling the new module
8. Installing the new module
9. Making EMC2 use the module

IMPORTANT ==========

This is not something you can just try if your latency is high.

You must establish that you have an Intel chipset which is amongst those affected AND that you are getting periodic regular spikes (32 or 64 seconds apart is common)

The chipsets in the 2.6.32-122-rtai realtime module are

82801AA_0
82801AB_0
82801BA_0
82801BA_10
82801E_0
82801CA_0
82801CA_12
82801DB_0
82801DB_12
82801EB_0
ICH6_0
ICH6_1
ICH6_2
ICH7_0
ICH7_1
ICH8_4

(Also ICH10_1 has been introduced in the later versions of magma, but to use this you would have to be comfortable with compiling your own module)

Run lspci -vv from a terminal to determine your chipset, the revision number in brackets is normally the match to the _N portion of the list above, eg ICH6 (revision 1) = ICH6_1

1. Intro

If you have a fairly new motherboard with Intel chipset (e.g. Intel 82845 845 (Brookdale) is a very common one), it most likely has SMI (System Maintenance Interrupt). SMI is a real-time killer and to avoid latencies of several 100 microseconds up to a few milliseconds, you need to disable them.

This issue is known by the RTAI people, as described in [[README.SMI]]

This document describes the needed steps to turn off SMI.

2. Turning off SMI may be dangerous

Disabling SMI can cause some computers to overheat and may damage those computers. Intel developer documentation states that running their modern processors with SMI disabled is running their processors outside of thermal spec, just like running the CPU with inadequate cooling is. Turn off SMI at your own risk.

3. Typical SMI interrupt

Typical SMI interrupt, seen while running the testsuite/kern/latency example of RTAI 3.3:
RTD|      -1234|      -1345|        583|       2206|       9884|          0
RTD|      -1209|      -1345|        575|       2098|       9884|          0
RTD|      -1206|      -1345|        591|     236277|     236277|          2
RTD|      -1261|      -1345|        609|       5581|     236277|          2
RTD|      -1274|      -1345|        579|       2468|     236277|          2
The SMI interrupt usually is very periodic, so if you let the latency test run for a while, you'll see it repeats. It has been reported that one value for the period is exactly 64 seconds.

RTAI comes with a kernel module, which disables all or certain SMI interrupt. This is included in the Hardy and Lucid EMC2 builds, but you will need to compile it yourself or download the binary for Dapper (Does anyone still use EMC2 on Dapper?)

4. Configuring SMI on Hardy or Lucid (Ubuntu 8.04 or 10.04, any LiveCD? installation newer than about 2008 (emc2-2.3.0))

To make EMC actually load the module at startup, you need to edit a couple lines in /etc/linuxcnc/rtapi.conf (Or /etc/emc2/rtapi.conf on versions < 2.5). With a Run-in-place compiled version you will need to find a version in /usr/local/etc/{linuxcnc / emc2} instead. You can work out which one is being used by renaming them to see which one stops LinuxCNC starting.)

rtapi.conf
#List of realtime kernel modules to be loaded
if [ "3" = "3" ] ; then \ 
    MODULES="adeos rtai_hal rtai_ksched rtai_fifos rtai_shm rtai_sem rtai_math rtai_smi"
    RTAI=3
    MODPATH_adeos=
    MODPATH_rtai_smi=/usr/realtime-$(uname -r)/modules/rtai_smi.ko
    MODPATH_rtai_hal=/usr/realtime-$(uname -r)/modules/rtai_hal.ko
    MODPATH_rtai_ksched=/usr/realtime-$(uname -r)/modules/rtai_ksched.ko
    MODPATH_rtai_fifos=/usr/realtime-$(uname -r)/modules/rtai_fifos.ko
    MODPATH_rtai_shm=/usr/realtime-$(uname -r)/modules/rtai_shm.ko
    MODPATH_rtai_sem=/usr/realtime-$(uname -r)/modules/rtai_sem.ko
    MODPATH_rtai_math=/usr/realtime-$(uname -r)/modules/rtai_math.ko

You need to add 'rtai_smi' into the list of modules, and add the extra MODPATH line.

When next you start EMC2, rtai_smi should be loaded:

$ lsmod | grep rtai
rtai_smi                2444  0
rtai_math              30468  0
rtai_sem               16640  1 rtapi
rtai_shm                9092  1 rtapi
rtai_fifos             26060  1 rtapi
rtai_sched             77848  4 rtapi,rtai_sem,rtai_shm,rtai_fifos
rtai_hal               32464  5 rtapi,rtai_sem,rtai_shm,rtai_fifos,rtai_sched
$

The above works for installed emc2 - for 'run in place' it is a bit different.

Copy the rtai_smi.ko to the correct location shown above

You need to edit the /your/rip/directory/scripts/rtapi.conf.in and make it look like

   
if [ "@RTAI@" = "3" ] ; then \     
    MODULES="adeos rtai_hal rtai_ksched rtai_fifos rtai_shm rtai_sem rtai_math rtai_smi"
    RTAI=@RTAI@
    MODPATH_adeos=@MODPATH_adeos@
    MODPATH_rtai_smi=/usr/realtime-2.6.15-magma/modules/rtai_smi.ko
    MODPATH_rtai_hal=@MODPATH_rtai_hal@

Then go into the /your/rip/directory/src
Make

It should now all work.

5. configuring SMI.ko on Dapper or any other installation where it is not present by default

You have to set the SMI options before you compile RTAI, or compile the SMI disabling module individually as described further.
You can either follow steps 6 and 7 and build a new module, or grab the binary (for Dapper only) from [[here]].

6. Changing smi-module.c

First you need to get the sources for the dapper version of RTAI:

to get the sources for the dapper version you need to
apt-get source rtai-3.3

Find the smi-module.c inside rtai/base/arch/i386/calibration, and change the following line:

#define CONFIG_RTAI_HW_SMI_ALL 1

Once that is changed, you can compile the module and replace the version already installed.

7. Compiling the new module

Unless you want to use the existing build process (which means you need to put the rtai sources in /usr/src/modules/rtai-3.3/ and use make-kpkg to build the modules) use the following Makefile:

Makefile
UNAME := $(shell uname -r)
KERNEL26 := 2.6
KERNELVERSION := $(findstring $(KERNEL26),$(UNAME))

obj-m	:= rtai_smi.o
rtai_smi-objs := smi-module.o

INCLUDE	:= -I/usr/include/asm/mach-default/
KDIR	:= /lib/modules/$(shell uname -r)/build
PWD		:= $(shell pwd)

all::
	$(MAKE) -C $(KDIR) $(INCLUDE) SUBDIRS=$(PWD) modules

Simply run make, and the new module should be built. (Alternative: grab the binary (for Ubuntu 6.06-Dapper Drake only) from [[here]]).

8. Installing the new module

Installing means simply copying the new binary module over the old version:

 sudo cp rtai_smi.ko /usr/realtime-2.6.15-magma/modules/rtai_smi.ko
(change 2.6.15-magma if your system uses a different kernel version)

9. Making EMC2 use the module

Follow the instructions for Hardy / Lucid in Step 4 above.


LinuxCNCKnowledgeBase | RecentChanges | PageIndex | Preferences | LinuxCNC.org
This page is read-only. Follow the BasicSteps to edit pages. | View other revisions
Last edited April 14, 2013 1:52 am by ArcEye (diff)
Search:
Published under a Creative Commons License