Linux VME Howto

John Huggins and Michael Wyrick, vmelinux@va.net

$Revision$, $Date$
This document came about to show the embedded system community how to run Linux on their VMEbus Pentium and other PCI local bus based VMEbus processor designs. The latest version is always available at Linux VME HOWTO.

1. Introduction

1.1 Knowledge Required

Using Linux on an embedded VMEbus processor board is not difficult. However, more than fundamental knowledge is required. This document is not a primer on how to fully configure a Linux machine.

In order to understand this HOWTO document it is assumed that you are thoroughly familiar with the following:

If you are uncertain of how to proceed with any of the above it is STRONGLY recommended that you use the links provided to familiarize yourself with all packages. We may not reply to any mail regarding any of the above. Please direct any questions to the appropriate author of the HOWTO or consult the respective hardware manufacturer.

This document describes the installation and use of VMELinux on a Xycom XVME-655 6U VME processor board. Other brands of VME boards that use a Pentium and the Tundra Universe chip should be capable of running VMELinux. Please consult the Board Support Section of the VMELinux web site for tested boards. VMELinux Project Web Site

1.2 Why use Linux on VMEbus systems?

Operating systems for VMEbus computers are usually Real-time Operating Systems (RTOS) which have high cost and a significant learning curve. In return the RTOS offers quick response to real world events for control of machinery or response to a process.

The VMEbus provides a rugged computer enclosure and interconnection system. Many system integrators require this ruggedness and also need very fast real-time response. However, there are many times when there is little need for real-time response, but the software still needs:

Linux has all these capabilities. Thus, the VMELinux Project exists.

1.3 Purpose

The purpose of VMELinux is to give the VME system integrator another choice in operating systems. Rich in features, high in reliability and low in cost, Linux offers benefits to the embedded computer industry. High cost operating systems economically prohibit the use of VME in many applications. With Linux and the VMELinux drivers, the rugged VMEbus has new possibilities.

The purpose of the VMELinux Project is to:

1.4 Feedback

As VMELinux is tested in the field, we encourage comments about how well or how bad it works. Please feel free to send comments to The VMELinux Project

As we get experience about each brand of VME CPU, we will list the different configurations in this HOWTO. For now we will describe only the Xycom board.

1.5 VMELinux Revision History

This document's revision is $Revision$, $Date$.

The latest version is always available at Linux VME HOWTO.

Linux Kernel Driver

VMEUtils Program

VMEShell Utilities

1.6 Copyright/Distribution

This document is Copyright 1997-2002 by John Huggins and the VMELinux Project.

A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similarly permitted without express permission if it includes a notice on who translated it. Commercial redistribution is allowed and encouraged; however please notify The VMELinux Project of any such distributions.

Excerpts from the document may be used without prior consent provided that the derivative work contains the verbatim copy or a pointer to a verbatim copy.

Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies.

In short, we wish to promote dissemination of this information through as many channels as possible. However, we wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO.

2. Installation of the VMELinux Kernel Driver

2.1 Download the Source

Download the distribution from the VMELinux Web Site.

2.2 Install the source to the software

Place the file in your source directory; We suggest /usr/src. Untar the zipped/tarred file by typing...

tar -xzf VMELinux_1.3.x.tar.gz

Then:

cd vmelinux
You should see three directories:
ca91c042
vmeshell
vmeutils

In ca91c042 you should find:

ca91c042/
ca91c042/Makefile
ca91c042/ca91c042.c
ca91c042/ca91c042.h
ca91c042/README
ca91c042/e
ca91c042/ins
ca91c042/stat
ca91c042/uns

In vmeshell you should find:

vmeshell/vmer
vmeshell/README
vmeshell/vmeseek
vmeshell/cmd.vme
vmeshell/vmew
vmeshell/vmeregw
vmeshell/vmeregr
vmeshell/vmefa
vmeshell/vmecall
vmeshell/e
vmeshell/ec
vmeshell/fa.vme
vmeshell/map.vme
vmeshell/tmp.vme
vmeshell/vmedb
vmeshell/vmedl
vmeshell/vmedw
vmeshell/vmemap
vmeshell/vmerb
vmeshell/vmerf
vmeshell/vmerl
vmeshell/vmerw
vmeshell/vmewb
vmeshell/vmewf
vmeshell/vmewl
vmeshell/vmeww
vmeshell/makelinks

In the vmeutils directory you should find:

vmeutils/commands.cpp
vmeutils/commands.h
vmeutils/universe.h
vmeutils/Makefile
vmeutils/vmeutils.h
vmeutils/unilib.h
vmeutils/unilib.cpp
vmeutils/vmeutils.cpp
vmeutils/README

2.3 Compile the VMELinux components

Enter the "ca91c042" directory and make the VMELinux device driver module.

make

Now you must create the several /dev driver files. Type:

make devices

DON'T FORGET TO MAKE THE /dev/vme* DEVICES!!!

Once made, you should see the file "ca91c042.o" in the directory. This is a loadable module. See below for loading information. Plus, you should find several "vme..." files in the /dev directory. Here is how the files should look:

hostname:/dev# ls -l vme*
crw-rw-rw-   1 root     root      221,   8 Jul 30 10:51 vme_ctl
crw-rw-rw-   1 root     root      221,   0 Jul 30 10:51 vme_m0
crw-rw-rw-   1 root     root      221,   1 Jul 30 10:51 vme_m1
crw-rw-rw-   1 root     root      221,   2 Jul 30 10:51 vme_m2
crw-------   1 root     root      221,   3 Jul 30 10:51 vme_m3
crw-rw-rw-   1 root     root      221,   4 Jul 30 10:51 vme_m4
crw-rw-rw-   1 root     root      221,   5 Jul 30 10:51 vme_m5
crw-rw-rw-   1 root     root      221,   6 Jul 30 10:51 vme_m6
crw-------   1 root     root      221,   7 Jul 30 10:51 vme_m7
hostname:/dev#

Change to the "vmeutils" directory and type make there.

make

This will compile the "vmeutils" program. This program directly speaks to the kernel driver. It is a reference work for those of you who wish to write your own programs to directly speak with the driver.

Copy the program "vmeutils" to your user binary directory or let the makelinks script do this for your in the next step. On our system this is "/usr/local/bin." Alternatively, you can create a link in the user bin directory to the "vmeutils" program.

Change to the "vmeshell" directory. There are no files to be compiled here. These are shell programs that use the "vmeutils" program to access the VMEbus. All the files beginning with "vme" should with have a link made or be copied to the "/usr/local/bin" directory.

just type:

./makelinks

Have a look in the libvme directory for a C++ example on how to communicate with the driver. You can use the libvme code as your interface to the driver for your programs if you wish. Docmentation for this is planned for the future.

You are now ready to try the driver.

2.4 Load the VMELinux Kernel Module

Make sure you are root and insert "load" the VMELinux Kernel Module for the Universe chip by typing...

insmod ca91c042

Or just type "./ins" to let the shell script do this for you. Once complete, type...

./stat

or

more /proc/ca91c042
You should see a list of registers displayed on your screen. Something like this...
Universe driver info:
  Control Pointer = 0000
  Stats  reads = 0  writes = 0  ioctls = 0
  LSI0_CTL = 00800000    LSI1_CTL = 00800000
  LSI0_BS  = C0000000    LSI1_BS  = 00000000
  LSI0_BD  = C0010000    LSI1_BD  = 00000000
  LSI0_TO  = 40009000    LSI1_TO  = 00000000
  LSI2_CTL = 00800000    LSI3_CTL = 00800000
  LSI2_BS  = 00000000    LSI3_BS  = 00000000
  LSI2_BD  = 00000000    LSI3_BD  = 00000000
  LSI2_TO  = 00000000    LSI3_TO  = 00000000
  image_va0   = 00000000     image_va1   = 00000000
  image_va2   = 00000000     image_va3   = 00000000

Driver Program Status:
  DMACTL 0    = 00000000 DMACTL 1    = 00000000
  DMACTL 2    = 00000000 DMACTL 3    = 00000000
  OkToWrite 0 = 0        OkToWrite 1 = 0
  OkToWrite 2 = 0        OkToWrite 3 = 0
  Mode 0      = 0        Mode 1      = 0
  Mode 2      = 0        Mode 3      = 0
If not, something went wrong.

2.5 Difficulties

The Universe driver does a good job of finding the Universe chip on a PCI bus, but differences in board design may prevent this. We tested all our routines on a Xycom XVME-655, Dynatem DPC and SBS VP7. There is little reason why this should not work on any other Intel board with a PCI bus and the Universe PCI-VME bridge chip. If you encounter problems, please let us know at the The VMELinux Project Bug Reporter

3. How to talk to the VMEbus with the VMEUtils and the VMEShell Packages

3.1 What is the VMEUtils program

This program can be run as is. Once started, you will see a command prompt. Type ? And you will see a list of commands. While useful, I think you will find the VMEShell scripts a better way to go. They do use this program to speak with the kernel driver so it is necessary to have this program available in the current PATH.

The source code for "vmeutils" is also instruction on how to speak directly to the kernel driver. For those of you who wish to create programs that directly speak with the driver, these source files are good examples.

3.2 What are the VMEShell Scripts

The VMEShell programs are unix shell scripts. They offer the operator a simple way to access the data on a VMEbus. Using these commands creates temporary files in the user's working directory which store information on the last access you did. This is nice because it will be possible to log off the machine, log back in and proceed from where you left off without having to re-enter VMEbus information again. Plus, these files are stored in the current working directory, so you can have different VME access configuration just by setting up different directories for each VME board of interest.

Assuming you placed the shell programs and the "vmeutils" program in the /usr/local/bin directory, you should be able to log in as a regular user and run them. What follows assumes exactly this.

3.3 The "vmemap" command.

Login as a regular user and create a directory to experiment with. Once in this directory type:

vmemap
You should get a help screen like this...
Usage:  map address count space size type
  where address is VME Address to set Universe image to

    Space = 0 CR/CSR    Space = 1 A16
    Space = 2 A24       Space = 3 A32

    Size  = 1 8 bit     Size  = 2 16 bit
    Size  = 3 32 bit    Size  = 4 64 bit

    Type  = 0 USR/DATA  Type  = 1 USR/PRG
    Type  = 2 SUP/DATA  Type  = 3 SUP/PRG
This is where you tell VMELinux how you want to access the VMEbus. We assume you already know about the VMEbus' many modes of operation, but here is a short list to help you. Typing...
vmemap 0x8000 0x100 1 2 0
sets up the VMELinux driver to access an A16 board at base address 8000 Hex with a range of 100H bytes with 16 bit data width and USR/DATA mode.

You will find two new files in your current directory.

fa.vme stores a "fixed adder" value that will be added to all subsequent accesses with the programs below.

map.vme store the parameters above so you do not have to enter them every time.

All the following shell utilities read values from these two files to performs VME accesses.

3.4 Read Byte, Word or Long

Syntax:

3.5 Write Byte, Word or Long

Syntax:

3.6 Read the VMEbus to a file

Syntax:

3.7 Write a file to the VMEbus

Syntax:

3.8 Parameters

There are several parameters used with these commands: address, size and filename.

3.9 Options

Available options are defined with a single dash with the any combination of the following:

3.10 A Note about DMA mode.

VMELinux offers access to all the features of the Universe Chip. Especially useful is access to the DMA engine on the chip. With this feature the Universe chip transfers data on the PCI bus by becoming a PCI master. This is nice, but the real benefit comes from the VMEbus accesses. Even if the VMEbus interface is not using block mode transfers, the Universe chip can complete VMEbus transfers under 400 nanoseconds sustained. This is the direct result of the Universe taking complete control of both the PCI bus and the VMEbus. Thus, it is possible to access non block mode VMEbus peripherals much faster than older technologies.

4. How to talk to the Tundra Universe PCI-VME bridge using the devices drivers.

4.1 The device drivers used with VMELinux

4.2 VMEMaster Device Drivers

/dev/vme_m* are drivers used to access the VMEbus as a bus master.

The Universe chip offers the programmer eight VMEMaster windows to the VMEbus. These windows are called Images. The details of the registers within these windows is beyond the scope of this Howto. Please refer to the Universe documentation for details. Tundra Universe

Version 1.1 of our tools only supported the first four images. This is because we originally designed this to work with the original Universe device. When the Universe II became available, Tundra did not update their documentation. Thanks to reports from other Universe users we are now aware of the new images, have found and downloaded the latest Universe manual from Tundra and have added these images to the 1.3.0 release.

4.3 VMESlave Device Drivers

The Universe chip offers the programmer four (eight for the UniverseII) VMESlave windows to the VMEbus. These windows are called Images. The details of the registers within these windows is beyond the scope of this Howto. Please refer to the Universe documentation for details. Tundra Universe

We originally intended to support the Universe's slave mode. We never had a need for this thus our efforts concentrated solely on using the Universe as a VME master only. So for 1.3.0 and the near future, we will not support the eight slave images.

I'll repeat this for clarity. Slave VME modes are not yet supported by our VMELinux Universe Kernel driver.

4.4 Direct Control of the Universe Registers

/dev/vme_ctl allows read and write access to the Universe registers.

For experienced users, this device allows direct access to the Universe chip's internal registers. Explanation of these registers and what they do is beyond the scope of this howto. Please consult the Universe documentation available from Tundra Universe

4.5 read()

n = read(vme_handle,buf,len);

Where:

4.6 write()

write(vme_handle,buf,len);

Where:

4.7 lseek()

lseek(vme_handle,vme_pnt,Seek_Type);

Where:

4.8 ioctl()

ioctl(vme_handle, command, argument);

Where:

And:

4.9 open() and close()

Here is where you open and close the four VMELinux Master or Slave devices plus the Control device. Slave images are not yet supported.

5. Advantages of the VMEbus, Linux and VMELinux

5.1 Pin and socket connectors

The VMEbus standard uses pin and socket connectors. This is superior to edge connections in that the connection is not exposed to humidity and other environmental conditions. It is a more expensive way of doing things, but offers longer times before failure.

5.2 Eurocard assembly

A VME board is either a 3U (160 x 100 mm) or a 6U size (160 x 233.35 mm). These sizes correspond to the Eurocard standard for board modules and card cages. Eurocard is a popular format used by many different busses including CompactPCI. This popularity makes the materials needed for cage assembly inexpensive and easy to obtain.

5.3 Linux is Low Cost

The nature of Linux is in its user supported and freely available format. The number of people using Linux is growing. The number of people contributing to the continued development of the Linux software base is growing. It is unfair to state that Linux is a good value because it is available for little to no charge. Linux is a good value because it works.

5.4 Linux is Stable

There are those who say that Linux us an unstable operating system. It is true that the new Linux kernels in development are experimental and should not be relied on for critical applications. However, stable versions of the Linux OS are always available and provide very robust operation. VMELinux is always based on the stable versions of the kernel source; Today's stable kernels are the 2.0.X and 2.2.x series. We have received reports that the latest 2.4.x kernels appear to be solid. I would say the future is plenty bright for Linux.

5.5 Linux is Dynamic

Because so many people are developing Linux, you do not have to wait long for improvements, fixes or new features to become part of the Linux distribution.

6. Current and planned Board Support

While the VMELinux driver should work with any PCI based design, the following boards have actually run our software.

6.1 Xycom XVME655 Pentium VMEbus Board

6.2 XyCom XVME656 Pentium VMEBus Board

6.3 Dynatem DPC1-0367

6.4 SBS/Or Computer VP7

6.5 DY4 179, A Power PC board

6.6 Planned Board Support

If you do not see VMELinux support for your board let us know. Maybe the manufacture will lend us a board for development.

7. Other "Linux on VME" Projects

This HOWTO emphasizes the efforts of just one particular way of accessing the VMEbus from a Linux system; Our way requires the Tundra Universe PCI/VME bridge device which will not work with many VME processor boards. Fortunately, there are several other efforts out there in various stages of development which provide the VME system integrator with options.

Since it is our desire to make this HOWTO reflect the efforts of the entire community of VME folks, we will be adding coverage of the other projects in future versions of this document. For the moment, we are simply going to list the other efforts in this section. Please refer to the latest documentation at The VMELinux Project for up to date information.

7.1 Project List

7.2 Major Device Numbers

There has been some confusion about the major device number to assign VME bus devices. Originally, the VMELinux Universe driver used 70. This quickly came into conflict with the "SpellCaster Protocol" as the number became assigned by the Linux folks. I requested and received a device number of 221 for VME devices. In a perfect world, all Linux VME design efforts would have a common interface to their driver through this device. I doubt we will ever see unity on this particular aspect, however, I think we can all at least agree to use this number for our devices.

Up to version 1.2.0 The VMELinux driver supports the following devices:

As of version 1.3.0 The VMELinux driver drops support for the slave images (it never did support them) and substitutes the four additional master images offered by the Universe II:

The good folks responsible for organizing Linux devices suggest the following device organization:

This was established from our 1.2.0 and earlier collections and makes sense for the Universe I device. For the Universe II and the many other completely different ways to the VMEbus, it makes no sense at all. I may ask the Linux folks to further breakdown the device tree like this:

All this is nice I suppose, but we like our devices to be /dev/vme* so our make file creates them in /dev. So far, the term "VME" has remained a unique identifier so conflicts with other devices should not occur; However, we should all remain watchful. So long as we all agree to use the major number of 221, all should be just fine. How we define the minor numbers does not need to be (and really cannot be) the same as other Linux-VME projects. However, this should not result in any conflicts in a particular installation. After all, one Linux VME system is not going to have more than one way to access the VMEbus.

Refer to the kernel.org web site for more details on this and every other assigned Linux device major number.

8. Conclusion

VMELinux and the other Linux on VME efforts offer the user a low cost way to implement a VMEbus system quickly, reliably and with all the advantages of a unix environment. We are using VMELinux in our projects. Our task list includes:

This document outlines the steps you need to install the VMELinux Kernel Driver into the example Xycom XVME-655 Pentium VME board. It is our hope that others will attempt installation of VMELinux into other boards and let us know their success.

This method of code development works best when the users tell us of their successes and describe the equipment used. Please, please drop a note to the VMELinux Mail List and share your experience with others.

Send bug reports and feature requrests to the VMELinux Project Bug Tracker. If you have a question or an update to this document send email to John.

9. FAQ

9.1 The Shell utilities return a bunch of stars (*) when I access a board I know is there. What gives?

Check to be sure the /dev/vme... files have their permissions set to 666. If not, the shell utilities will return a * in place of data to indicate an error condition similar to a VME bus error.

9.2 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. Now what?

It is possible the ca91c042 Linux kernel module has been compromised. Get root access and type "lsmod" to review the loaded modules. Do you see the ca91c042? If yes, try removing it and reinstalling it with "rmmod ca91c042" and then "insmod /path/to/the/ca91c042.o" to get things up again. If it is not there check to see if you are loading the module when you boot the machine, etc.

9.3 The Shell utilities still return a bunch of stars (*) when I access a board I know is there. HELP?

Time to get a VMetro board into the VME cage and see if any accesses are occurring. Also look at the /proc/ca91c042 file to see if the read and write counters are incrementing.

9.4 How does VMELinux handle interrupts?

The driver does handle interrupts, but if you compile your interrupt handler program as a Linux loadable module, that program can handle the interrupts directly. Examples of this will be available soon. It is important to note that user level program can be made to handle interrupts, but it is a much better idea to have your interrupt handlers as part of the Linux kernel via loadable modules. Yes, you can totally hose the kernel if you do something wrong, but that is the trade off between safety and performance.

9.5 I have RedHat 5.1 and can't get VMELinux programs to compile.

RedHat 5.1 includes a new compiler. If you manually edit the Makefile in each directory to call up the new egcs compiler, things should compile. We fully intend to support RedHat 5.1 installations, but for now I suggest using 5.0 or Slackware.

9.6 I have RedHat 6.x so I assume the above issue is fixed. Right?

Maybe. RedHat threw us, and many other kernel module driver writers, a curve ball with the move to the egcs compiler. Thankfully, the two compiler camps, GCC and egcs, have united their efforts. All this incompatibility should just go away. For the moment, however, VMELinux will only be tested with GCC 2.95.x so that is what we suggest you use for a compiler. If you type "gcc --version" at your prompt and get an "egcs..." back then we cannot say it will work for you.

9.7 When will your ca91c042 Tundra Universe driver support the 2.4 kernels?

Now. Download the latest tar ball from the download directory at the main site. 2.4 support was added in version vmelinux-1.2.0.

9.8 Hey! The Universe II has eight master and eight slave images. You support four each. Why?

We have been at this a long time and initially created these tools for the original Universe I which according to the documentation has four images each. When boards arrived with the Universe II, we searched the Tundra web site in vain for new documentation. We were told by the board manufacturers the II works just like the original; Thus, we only worried about the original four images. Just recently some good folks pointed out our omission. We finally found the correct documentation on the web site and support the extra images as of the 1.3.0 development release.

More news! The latest CVS snapshots of the ca91c042.c and ca91c042.h files will notice which Universe version is in your system and act accordingly.

Please note, we have not yet found any good reason to spend time developing support for the eight (or four) slave images. The current tools only support the master images and this has proven adequate for every need we have come across. If you think slave support is a good thing, let us know.

The folks at VMIC have a kernel driver they say does support the slave images and is available under a BSD style license. The announcment may be found in our mail list archive mail list archive and the correct link to there web site is here.

9.9 How can we contribute to your VME-LINUX working group?

All programming efforts thus far have been accomplished by Michael Wyrick. My role has been to coordinate the VMELinux Project and help define and test the resulting programs.

We have successfully placed the working code into a CVS system and are using it to track code changes. Right now only Mike and myself have write access to this. If you are seriously interested in VMELinux development and you know your way around Linux kernel programming, please join us by joining the developers mailing list and creating an account on our bug tracking system located here.

If you cannot develop code, please consider keeping us informed about any bugs you see or features we should add. You can send mail to the user or developer mail lists, but contributing your comments to the bug tracking system is much more useful. Just visit VMELinux Project Bug Tracking System, create an account, submit your report and we will address it as soon as we can.