STM32F407 DMA early tests

Research notes. Useful links:

DMA is a peripheral that can copy data between other peripherals and memory or between memory and memory. It used to be implemented in form of separate IC in early days, but in modern µCs it is of course integrated inside the single chip.

STM32 DMA peripherals are able to copy data from memory to peripheral, from peripheral to memory and from memory to the other place in memory (for example from RAM to FLASH as StdPeriph example shows). There are two DMA controllers : DMA1 and DMA2 and both have 8 streams. I see a stream as a some kind of physical, bi-directional connection between the DMA controller and some other peripheral. Those 16 streams cover all (most?) peripherals meaning that one stream is connected to more than one peripheral. For example if one is about to send data using USART1 he has to use exactly DMA2_Stream7, or if he wants to receive data from SPI3_RX he has to use DMA1_Stream0 or DMA1_Stream2, because apparently SPI3_RX is connected to both of those streams (see table 43 and 44 in reference manual of STM32F407).

DMA works automatically meaning that if there is some new data it will be copied without user code (and CPU) involved. It is possible thanks to channels, which I imagine as signals (like GUI signals if you know what I mean) connected between DMA controller and the peripheral (there is also something called “arbiter” between them). Peripheral can send a request (signal) to the DMA if it has new data, and DMA can then process it. At the same time DMA acknowledges it has got this new portion of data, and peripheral releases the request. Each stream can “listen” to its 8 channels, so there are 2 controllers * 8 streams * 8 channels = 128 configuration combinations, and that way every peripheral can have its own communication path with the DMA.

Streams have configurable priorities in case two or more streams request DMA controller attention. If two or more streams have the same priority, then stream with lower number wins. The bit of hardware called “arbiter” manages those priorities and decides which stream goes first.

So here comes the first DMA test I wrote (tested on STM32F407-DISCOVERY). It writes to USART1:

#include <stm32f4xx.h>
#include "logf.h"
 
/**
 * For printf, and USART1 in general.
 */
void initUsart (void)
{
        RCC_APB2PeriphClockCmd (RCC_APB2Periph_USART1, ENABLE);
        GPIO_InitTypeDef gpioInitStruct;
 
        RCC_AHB1PeriphClockCmd (RCC_AHB1Periph_GPIOB, ENABLE);
        gpioInitStruct.GPIO_Pin = GPIO_Pin_6 | GPIO_Pin_7;
        gpioInitStruct.GPIO_Mode = GPIO_Mode_AF;
        gpioInitStruct.GPIO_Speed = GPIO_High_Speed;
        gpioInitStruct.GPIO_OType = GPIO_OType_PP;
        gpioInitStruct.GPIO_PuPd = GPIO_PuPd_UP;
        GPIO_Init (GPIOB, &gpioInitStruct);
        GPIO_PinAFConfig (GPIOB, GPIO_PinSource6, GPIO_AF_USART1); // TX
        GPIO_PinAFConfig (GPIOB, GPIO_PinSource7, GPIO_AF_USART1); // RX
 
        USART_InitTypeDef usartInitStruct;
        usartInitStruct.USART_BaudRate = 9600;
        usartInitStruct.USART_WordLength = USART_WordLength_8b;
        usartInitStruct.USART_StopBits = USART_StopBits_1;
        usartInitStruct.USART_Parity = USART_Parity_No;
        usartInitStruct.USART_Mode = USART_Mode_Rx | USART_Mode_Tx;
        usartInitStruct.USART_HardwareFlowControl = USART_HardwareFlowControl_None;
        USART_Init (USART1, &usartInitStruct);
        USART_Cmd (USART1, ENABLE);
}
 
uint8_t myStrlen (char const *s)
{
        uint8_t len = 0;
 
        while (*s++) {
                ++len;
        }
 
        return len;
}
 
/**
 * Test1
 */
void initDma (char const *outputBuffer)
{
        /*
         * Reset DMA Stream registers (for debug purpose). For DMA2_Stream7 exmplanation read on.
         * It also disables the stream. Stream must be disabled prior configure it. Otherwise it can
         * misbehave.
         */
        DMA_DeInit (DMA2_Stream7);
 
        /*
         * Check if the DMA Stream is disabled before enabling it.
         * Note that this step is useful when the same Stream is used multiple times:
         * enabled, then disabled then re-enabled... In this case, the DMA Stream disable
         * will be effective only at the end of the ongoing data transfer and it will
         * not be possible to re-configure it before making sure that the Enable bit
         * has been cleared by hardware. If the Stream is used only once, this step might
         * be bypassed.
         */
        while (DMA_GetCmdStatus (DMA2_Stream7) != DISABLE) {
        }
 
        /* Configure the DMA stream. */
        DMA_InitTypeDef  dmaInitStructure;
 
        /*
         * Possible values for DMA_Channel are DMA_Channel_[0..7]. Refer to table 44 in reference manual
         * mentioned earlier. USART1_RX is communicate with DMA via streams 2 and 5 (both on channel 4).
         * USART1_TX uses stream7 / channel 4.
         */
        dmaInitStructure.DMA_Channel = DMA_Channel_4;
 
        /*
         * Possible values : DMA_DIR_PeripheralToMemory, DMA_DIR_MemoryToPeripheral,
         * DMA_DIR_MemoryToMemory.
         */
        dmaInitStructure.DMA_DIR = DMA_DIR_MemoryToPeripheral;
 
        /* Why DMA_PeripheralBaseAddr is of type uint32_t? Shouldn't it be void *? */
        dmaInitStructure.DMA_PeripheralBaseAddr = (uint32_t)&(USART1->DR);
        dmaInitStructure.DMA_Memory0BaseAddr = (uint32_t)outputBuffer;
 
        /*
         * Only valid values here are : DMA_PeripheralDataSize_Byte, DMA_PeripheralDataSize_HalfWord,
         * DMA_PeripheralDataSize_Word
         */
        dmaInitStructure.DMA_PeripheralDataSize = DMA_PeripheralDataSize_Byte;
 
        /*
         * I guess, that for memory is is always good to use DMA_MemoryDataSize_Word (32bits), since this is
         * a 32 bit micro. But haven't checked that. But here I use Byte instead for easier  DMA_BufferSize
         * calculations.
         */
        dmaInitStructure.DMA_MemoryDataSize = DMA_MemoryDataSize_Byte;
 
        /*
         * Length of the data to be transferred by the DMA. Unit of this length is DMA_MemoryDataSize when
         * direction is from memory to peripheral or DMA_PeripheralDataSize otherwise. Since I set both
         * sizes to one byte, I simply put strlen here.
         */
        dmaInitStructure.DMA_BufferSize = myStrlen (outputBuffer);
 
        /*
         * DMA_PeripheralInc_Disable means to read or to write to the same location everytime.
         * DMA_MemoryInc_Enable would increase memory or peripheral location after each read/write.
         */
        dmaInitStructure.DMA_PeripheralInc = DMA_PeripheralInc_Disable;
        dmaInitStructure.DMA_MemoryInc = DMA_MemoryInc_Enable;
 
        /* DMA_Mode_Normal or DMA_Mode_Circular here. */
        dmaInitStructure.DMA_Mode = DMA_Mode_Normal;
 
        /* DMA_Priority_Low, DMA_Priority_Medium, DMA_Priority_High or DMA_Priority_VeryHigh */
        dmaInitStructure.DMA_Priority = DMA_Priority_VeryHigh;
 
        /* DMA_FIFOMode_Disable means direst mode, DMA_FIFOMode_Enable means FIFO mode. FIFO is good. */
        dmaInitStructure.DMA_FIFOMode = DMA_FIFOMode_Disable;
 
        /*
         * DMA_FIFOThreshold_1QuarterFull, DMA_FIFOThreshold_HalfFull, DMA_FIFOThreshold_3QuartersFull or
         * DMA_FIFOThreshold_Full.
         */
        dmaInitStructure.DMA_FIFOThreshold = DMA_FIFOThreshold_Full;
 
        /*
         * Specifies whether to use single or busrt mode. If burst, then it specifies how much "beats"
         * to use. DMA_MemoryBurst_Single, DMA_MemoryBurst_INC4, DMA_MemoryBurst_INC8 or
         * DMA_MemoryBurst_INC16.
         */
        dmaInitStructure.DMA_MemoryBurst = DMA_MemoryBurst_Single;
        dmaInitStructure.DMA_PeripheralBurst = DMA_PeripheralBurst_Single;
 
        /* Configure DMA, but still leave it turned off. */
        DMA_Init (DMA2_Stream7, &dmaInitStructure);
 
        /* DMA_FlowCtrl_Memory, DMA_FlowCtrl_Peripheral */
        DMA_FlowControllerConfig (DMA2_Stream7, DMA_FlowCtrl_Memory);
 
        /* Enable DMA interrupts. */
        DMA_ITConfig (DMA2_Stream7, DMA_IT_TC | DMA_IT_HT | DMA_IT_TE | DMA_IT_DME | DMA_IT_FE, ENABLE);
 
/*--------------------------------------------------------------------------*/
 
        /* Enable the DMA Stream. */
        DMA_Cmd (DMA2_Stream7, ENABLE);
 
        /*
         * And check if the DMA Stream has been effectively enabled.
         * The DMA Stream Enable bit is cleared immediately by hardware if there is an
         * error in the configuration parameters and the transfer is no started (ie. when
         * wrong FIFO threshold is configured ...)
         */
        uint16_t timeout = 10000;
        while ((DMA_GetCmdStatus (DMA2_Stream7) != ENABLE) && (timeout-- > 0)) {
        }
 
        /* Check if a timeout condition occurred */
        if (timeout == 0) {
                /* Manage the error: to simplify the code enter an infinite loop */
                while (1) {
                }
        }
}
 
int main (void)
{
        /* This would be a function parameter or something like that. */
        char *outputBufferA = "Ala ma kota, a kot ma ale, to jest taki wierszyk z czytanki dla dzieci, ktora jest tylko w Polsce.\r\n";
        char *outputBufferB = "Wlazl kotek na plotek i mruga. Ladna to piosenka nie dluga. Nie dluga, nie krotka lecz w sam raz.\r\n";
 
        /*
         * Enable the peripheral clock for DMA2. I want to use DMA with USART1, so according to
         * table 44 in reference manual for STM32F407 (RM0090) this would be DMA2 peripheral.
         * Description in stm32f4xx_dma.c advises to do this as the first operarion.
         */
 
        /*
         * Spend two fu.king nights on this. Docs says to use RCC_AHB1PeriphResetCmd, but use
         * RCC_AHB1PeriphClockCmd instead!!!
         */
//        RCC_AHB1PeriphResetCmd (RCC_AHB1Periph_DMA2, ENABLE);
        RCC_AHB1PeriphClockCmd (RCC_AHB1Periph_DMA2, ENABLE);
 
/*--------------------------------------------------------------------------*/
 
        /*
         * Enable the USART1 device as usual.
         */
        initUsart ();
        logf("Init\r\n");
 
/*--------------------------------------------------------------------------*/
 
        initDma (outputBufferA);
 
        /*
         * The DMA stream is turned on now and waits for DMA requests. As far as I know, if this
         * were to be memory-to-memory transfer, it would start immedialtely without enabling any
         * channels. But for peripherals one has to enable the channel for requests. After following
         * statement, you should see data on serial console.
         *
         * This statement enables the DMA internals in USART (this stuff which communicates with the DMA
         * controller).
         */
        USART_DMACmd (USART1, USART_DMAReq_Tx, ENABLE);
 
        /* Waiting the end of Data transfer */
        while (USART_GetFlagStatus (USART1, USART_FLAG_TC) == RESET)
                ;
 
        while (DMA_GetFlagStatus (DMA2_Stream7, DMA_FLAG_TCIF7) == RESET)
                ;
 
        logf("It worked, and didn't hanged\r\n");
 
        /* Clear DMA Transfer Complete Flags */
        DMA_ClearFlag (DMA2_Stream7, DMA_FLAG_TCIF7);
 
        /* Th has to be initialized once again AFAIK to send another portion of data. */
        initDma (outputBufferB);
 
        /* Try to start it again */
        USART_DMACmd (USART1, USART_DMAReq_Tx, ENABLE);
 
        /* Waiting the end of Data transfer */
        while (USART_GetFlagStatus (USART1, USART_FLAG_TC) == RESET)
                ;
 
        logf("It workedagain\r\n");
 
        /* Infinite loop */
        while (1) {
        }
}

 

STM32F407-DISCOVERY SDIO tests.

  • Started a new project (includes StdPeriph 1.3.0). Repository can be found here.
  • First commit makes it simply output a “Init” text on the debug console (i.e. on USART1).
  • Browsing StdPeriph. Seems that stm32f4xx_sdio.[ch] are very low level (I’ve read SD card spec version 2.0).
  • Higher level stuff seems to be in StdPeriph here:
    • Utilities/STM32_EVAL/STM324x7I_EVAL
    • Utilities/STM32_EVAL/STM3240_41_G_EVAL
    • Utilities/STM32_EVAL/STM324x9I_EVAL

But don’t know why there is one version per dev-board. Looks like bad design to me at a first glance. Differences between those 3 files:

  • Lines 517/519 : Card presence is detected by different means (different pins are used on these boards). As far as I remember card presence detection is an optional feature, so maybe even it is not part of the standard. That would explain why different pins are used on the boards (I expect, that standard pins are laid out the same on …?).
  • Lines 1570/1572 different parameter passed to SDIO_ITConfig in function SD_WriteMultiBlocks. One dev-board uses SDIO_IT_RXOVERR and the other two useSDIO_IT_TXUNDERR (among others, this is a bitmask).
  • Copied STM324x9I_EVAL sdio routines to my source tree.
  • Included code from the SDIO example : Project/STM32F4xx_StdPeriph_Examples/SDIO/SDIO_uSDCard. Serial console went crazy and shows some gibberish, so I can’t see my debug messages, but card previously filled with random data now shows :
root@diora:~# hexdump -n 128 /dev/sdc 
0000000 0100 0302 0504 0706 0908 0b0a 0d0c 0f0e
0000010 1110 1312 1514 1716 1918 1b1a 1d1c 1f1e
0000020 2120 2322 2524 2726 2928 2b2a 2d2c 2f2e
0000030 3130 3332 3534 3736 3938 3b3a 3d3c 3f3e
0000040 4140 4342 4544 4746 4948 4b4a 4d4c 4f4e
0000050 5150 5352 5554 5756 5958 5b5a 5d5c 5f5e
0000060 6160 6362 6564 6766 6968 6b6a 6d6c 6f6e
0000070 7170 7372 7574 7776 7978 7b7a 7d7c 7f7e
0000080

Looks less random to me.

  • Had problems with serial console attached to USART1 after upgrading StdPeriph from 1.1.0 to 1.3.0. Console would speak Chinese from now on, and logic analyzer shows “framing errors” when attached to the TX pin. There were two problems: in 1.1.0 in file stm32f4xx.h default HSE_VALUE definition was 8MHz. In 1.3.0 ST increased this to 25MHz. In addition in stm32f4xx_conf.h I had HSE_VALUE redefined, but later on I upgraded this file (got it from some example projest from StdPeriph from 1.3.0 version), and it lacked this re-definition. Thus µC thought it is running on 25MHz which in turn disrupted the transmission. This re-definition looks as follows:
#if defined  (HSE_VALUE)
/* Redefine the HSE value; it's equal to 8 MHz on the STM32F4-DISCOVERY Kit */
 #undef HSE_VALUE
 #define HSE_VALUE    ((uint32_t)8000000) 
#endif /* HSE_VALUE */
  • I am facing a similar problem like this. Program hangs after returning from interrupt routine(?) In fact I don’t really know what is happening… Program hangs inSD_WaitReadOperation after successfully returning from SD_ReadMultiBlocks. It idles in loop (or at first glance it looks like it is iterating the loop forever) which looks like this:
while ((DMAEndOfTransfer == 0x00) && (TransferEnd == 0) && (TransferError == SD_OK) && (timeout > 0)) {
    timeout--;
}

Normally after successful transfer either DMAEndOfTransfer or TransferEnd would turn 1, but seemingly none of this happened. The only place the TransferEnd is set isSDIO_IRQHandler, so I added logs to check if µC hits this routine. It does, and it even set TransferEnd to 1, but it never returns from it. Debugger says, that program hangs in some strange places like WWDG_IRQHandler.

The problem was caused by missing DMA handler routine, namely the DMA2_Stream3_IRQHandler. I made two mistakes. First, I assumed, that since I run the demo with SD_DMA_MODE turned off (undefined), and SD_POLLING_MODE turned on (#defined), the DMA routines are unnecessary. This is not the case, those handlers are required in either cases (that’s the way SDIO example is made). So I copied DMA_IRQ handler from the example, where its name was hidden behind the SD_SDIO_DMA_IRQHANDLER macro (but at that point I didn’t know this is a macro, and thought that this is a regular function name). So secondly, SD_SDIO_DMA_IRQHANDLER was undefined in my stm32fxxx_it.c, It simply was not visible in this translation unit, and I ended up with function named SD_SDIO_DMA_IRQHANDLER, but without proper DMA IRQ handler. So µC jumped to the default handler which had infinite loop in it, but for some reason GDB showed the other handler.

A chaotic post on HID keyboard : STM32F407 success, STM32F105 fail

This is a quick dev-log post on my latest design, which was only partially successful. I have STM32F407-DISCOVERY board on which I successfully implemented a HID keyboard with only one keyboard. At first it reported that ‘a’ key was pressed every time user pressed the blue button, then, according to my plan I changed this to play/pause button, which can turn music on and of. It works under Linux and Windows (only ‘a’ version tested under win though). Then I decided to make a board for this and, since F407 is quite expensive, in fact too expensive for simple one-key keyboard, I decided to use something simpler. The slowest and cheapest µcros that support STM32_USB-Host-Device_Lib are those labeled as “connectivity line” i.e. STM32F105 and STM32F107. I’ve got myself two STM32F105R8T6 and made a board, which fits into a case I also bought. The case is labeled as “XB2-ES542”  :

boardthe case

Eagle schematic and board are here, OSH Park shared projest is here. I assumed (wrongly) that porting my program from F407 dev board to my custom board featuring different micro will be easy since they are quite similar. I was wrong. And I don’t have a dev board for F105 nor 107. Ok, but first things first. As I mentioned, program works on F407, so let me write down some random thoughts which emerged during the process of making this work:

Few facts about HID devices (that I learned)

All data exchanged resides in structures called reports. The host sends and receives data by sending and requesting reports in control or interrupt transfers. The report format is flexible and can handle just about any type of data, but each defined report has a fixed size. The device’s descriptors must include an interface descriptor that specifies the HID class, a HID descriptor, and an interrupt IN endpoint descriptor. The firmware must also contain a report descriptor that contains information about the contents of a HID’s reports.

So there are two additional descriptors when comparing to the ‘vendor specific’ device I made recently (there may be third, optional descriptor as well). First is HID class device descriptor and it specifies which other class descriptors are present (for example report descriptors or physical descriptors).

A HID can support one or more reports. The report descriptor specifies the size and contents of the data which this device generates. Physical descriptors at the other hand are optional pieces of data which describe the part(s) of human body used to operate the HID device. The HID class does not use subclasses to define most protocols. Instead, a HID class device identifies its data protocol and the type of data provided within its Report descriptor.

Here on page 53 you can find all key codes defined by the HID spec. Document “Device Class Definition for Human Interface Devices (HID) Version 1.11” on page 62 has very useful information regarding keyboard implementation. Especially crucial are those bits about when to send a data report: “The keyboard must send data reports at the Idle rate or when receiving a Get_Report request, even when there are no new key events.” I mixed up the rates and my HID keyboard acted unpredictable. Only after adjusting wait period for 4ms * idle rate things went OK.

Then I started to getting familiar with the report descriptors, but nah, the more I read HID specification the more I realized this is too much complexity, and too much effort than I wanted to put into this project. At first I was like, “OK let’s read the whole spec, it has 97 pages, I’ve read longer specs before, not a problem”. But hey, this was meant to be a simple, few evenings project, and this HID spec turned out to be surprisingly complex (I mean report descriptors in particular. When I came to Push and Pop items I refused to read further). The better and simpler way of accomplishing this project was to grab some descriptors from the net, and so I did:

  • Here I found useful report descriptor for regular keyboards (like qwerty ones). Other, special buttons are implemented by other means (other items in reports) as I noticed before.
  • Here are some interesting report descriptors which looks like something I want to do (multimedia control). Seems to me, that all those volume, play/pause and other knobs are implenmented in some other means that regular keyboards are. There are different type of “usage” used i.e. regular keyboard has 0x09 0x06 (USAGE (Keyboard)), but in above document there is 0x09 0x01 (Usage (Consumer Control)) used.

There is also a tool on USB-IF page, which helps to assembly HID report descriptors, and I can confirm that it runs on Wine, but that is all I can say about it. I suppose you still have to know the specs, and know what you are doing while using this thing. Last but not least the source code which runs flawlessly on STM32F407-DISCO:

Failed attempt to port it to STM32F105

Then I started to move to my custom board depicted above, which I didn’t managed to accomplish. Actual status of the whole device (source code linked below, Eagle files above) is that, after connecting the USB, device initializes itself (i.e. USB stack gets initialized) then it gets quite a few reset requests (like 10) and then it hangs. Wireshark + usbmon shows “Malformed packet” when device tries to send the Device descriptor to host. Random notes from development:

  • Note : STM32F105 and 107 are called “connectivity line microprocessors”. It is useful to know that since there are many resources for STM32F1x out there which are tagged like “value line”, “connectivity line” etc.
  • I copied my previous project stm32f407-drama-button into new place in my SVN repository : stm32f105-drama-button.
  • Downloaded and unpacked STM32F10x_StdPeriph_Lib_V3.5.0 library from here. Main page for this micro is here. Current version of standard peripheral library for STM32F10x as of writing this is 3.5.0.
  • Replaced /STM32F4xx_StdPeriph_Driver with /STM32F10x_StdPeriph_Driver.
  • Replaced CMSIS folder. Removed Docs folder.
  • Made new toolchain with crosstool-ng fine tuned for Cortex-M3 µC.
  • Copied and modified stm32f105-crosstool.cmake.
  • StdPeriph comes with ld-scripts for various dev-boards. I figuret out that :
  • So linker script for STM3210C-EVAL is best suited for me and will check it first. Have it copied and modified. In “drama button” project I use STM32F105R8T6 which has:
    • 64kB of flash,
    • 64kB of RAM
  • Copied stm32f10x_conf.h from examples into src. CMSIS uses it somehow and I think this is bad design. Lower level library depends on higher level header file?
  • Had trouble when used GCC-4.8.0 (linaro version made with ct-ng 1.19). Works fine with GCC-4.7.0 (linaro version prepared with ct-ng 1.18). Some strange assembler errors when compiling core_cm3.c file poped out. Here guy in comments had similar issue, and someone told him to download fresh CMSIS library, because this provided with StdPeriph is old. I can believe that, because for example ST USB OTG library comes with StdPeriph bundled inside and it is also some old version. So my rule of thumb now is to collect newest versions of all the individual libraries, even when they are distributed together (i.e. OTG library virtually has everything required to compile the examples it provides, but now I throw it away and get fresh ones).

EDIT : when compiling with gcc-4.7.0 with heavy optimizations (-O3) the same assembler error emerges. Message is

/tmp/ccOfylWN.s: Assembler messages:
/tmp/ccOfylWN.s:646: Error: registers may not be the same -- `strexb r0,r0,[r1]'
/tmp/ccOfylWN.s:675: Error: registers may not be the same -- `strexh r0,r0,[r1]'

Downloaded and upgraded CMSIS from here : https://silver.arm.com/browse/CMSIS# (registration required). Upgraded from version 1.3.0 to 3.2.0. No *.c file this time, only header files.

When ported from STM32F407-DISCO to my custom STM32F105 board, program refused to operate. Kernel log says that:

Jan  3 00:08:09 diora kernel: [ 4993.895502] usb 7-2: new full-speed USB device number 4 using uhci_hcd
Jan  3 00:08:09 diora kernel: [ 4993.959542] hub 7-0:1.0: unable to enumerate USB device on port 2

And oscilloscope says, that… well… seems OK to me (compared for example to Wikipedia article on USB) :)

map001 map002

Debugger shows that program is running, It correctly invoke Reset_Handler, and then hits the main function.

Mistakes that I know I made:

  • Made a circuit with µC that I don’t have development board for (Assumed, that porting tested program from STMF407 to STMF105 will be easy. Wrong).
  • Screwed up a SWD port. Issues with signal integrity (probably due to lack of termination resistors?). This made debugging with GDB harder. It would hang, or disconnect at random points.
  • Forgot to add serial console output for STDOUT. Terrible mistake. I’ve wired that up, but board looks messy now.
  • Simple delay functions was dependent on optimization switches.

And then, after about 10 evenings / nights I gave up and put this project on the shelf for some time.

Motorcycle black box. A quick update.


Just a quick update for those who are interested if this project is still alive. It is. Above you can find an early test video from the application which runs on a PC and does post processing of the data collected by raspberry pi. More details to come in next post. All source code for this project is here (work in progress). First post here.

Motorcycle black box. Part 1 : data acquisition with Arduino Mega.

The objective

Make a black box, a device which would record short clips from camera in loop overwriting oldest clips. That would produce constant stream of short movies which put together would make one long recording containing last 30minutes (or more – depending on storage capacity) of my riding. Among with the recording, telemetric data would be collected such as velocity, RPM, temperature, front and rear brake pressed, depressed, and turn signals, and maybe more. Then this data could be put on the video in overlay.

The motorcycle

Yamaha XJ6SA 2010.

Identify by what means communication between the ECU and the dashboard is performed.

Which cables are used and what is the protocol used? After inspecting the service manual for my bike I was able to eliminate wires which are used for other things (see picture), and I left with only one, which is connecting the dash, the ECU and the immobilizer. Other ones was for flashing some warning LEDs, gathering information from fuel pump, oil switch and so on. So the yellow-blue wire was my first guess and it was correct. On the ecuhacking forum, which was source of very helpful information, guys was talking about something called K-line. I believe it is something widely common in the automotive industry, and for certain it has bunch of ISO standards describing it, but hey, if this is only one wire, and data flowing inside is some kind of serial communication, I bet I could sniff it, and figure out without any standards, which are hard to find and get (there is so much of them, I’ve got confused after few minutes of googling).

Dashboard connections

Dashboard connections

Figure out how to interface this, make some circuit if needed.

Again on the ecuhacing forum I’ve found information that K-line has logic levels relative to 12V, where 0V is logic 0, and 12V logic 1. There has to be some voltage level converter if I want to connect some TTL stuff to it. On the forum some guys were using a L9637 chip which is described as “ISO 9141 INTERFACE” by its datasheet. So I bought a few of these and connected it as follows :

L9637

L9637

Later on I removed the 510 ohm pull up resistor, because after turning engine on there was error showing of on the dash. I guessed that could be this resistor, and it helped. Dunno what was wrong.

Most difficult part for me was to find a place in motorcycle’s wiring to connect to. After some time I managed to insert piece of rigid wire to the back side of ECU connector as depicted below:

ECU connector

ECU connector

5V supply voltage I was drawing from step down voltage converter which I bought on the Internets. It is a PCB with a few discrete parts and quite huge radiator with some coils inside. There is [EDIT] label on it. To the output of L9637 I connected a Saleae logic analyzer, which I also bought. It is quite cheap and has good software for Windows, Linux and MAC OS. I can definitely recommend it, especially if you want to use it with Linux (I use Ubuntu).

Sniff the data, and collect it for further investigation.

My circuit worked the first time. After turning ignition to the ON state this is what I got:

Sealae logic analyzer window

Sealae logic analyzer window

Sealae app, among others, has a “Async serial” analyzer which I used with default configuration, and “use autobaud” checkbox checked. This useful option corrected initial 9600 baud to whatever it thought to be appropriate, and showed 16064 baud. Pretty odd value isn’t it? None the less it is very useful information if I want to read the data with some AVR, or something like that. It turns out, that data comes in packets which are 6B long. First byte is some kind of command, and the rest is the reply i.e the dashboard issues command 0x01 ant then ECU replies with 5 bytes of data. From the ecuhacking I knew that forst byte of the reply would be the RPM, second probably velocity, third an error code, fourth would be engine temperature (that is coolant or oil temperature?) and the very last is the checksum.

Sealae async serial analyzer setup

Sealae async serial analyzer setup

Collect the data in some more useful format.

Although Sealae has an option to store the analyzer’s data in CSV, sooner or later I would have to make some custom electronics anyway. Yet I want to make self contained logger hidden in neat case somewhere in the bike. I want to use a Raspberry PI i own as the main component of the gizmo, but after reading about serial communication with the Pi, I gave up for that time, and connected Arduino Mega. The main problem with PI is that I don’t know how to set it up for so unusual baud rate as 16064 baud (or 15625 as someone suggested). I had limited time, so for now I chose the Arduino. Below the code I uploaded :

#include <SoftwareSerial.h>
 
SoftwareSerial mySerial(10, 11); // RX, TX
 
void setup() {
    // initialize both serial ports:
    Serial.begin(115200);
 
    // set the data rate for the SoftwareSerial port
    mySerial.begin(16064);
}
 
void loop() {
    static int count = 0;
 
    // read from port 1, send to port 0:
    if (mySerial.available()) {
        int inByte = mySerial.read();
 
        if (inByte == 0x01 && count >= 5) {
            Serial.println(' ');
            count = 0;
        }
 
        Serial.print(inByte, HEX);
        Serial.print (' ');
        ++count;
    }
}

As you can see, RX is set up for pin 10, so the only thing I did was to disconnect the logic analyzer, and connect Arduino pin 10 instead. But nothing happened. This was because the SoftSerial library have a few baud rates predefined, namely the most usual ones. The solution was to modify arduino-1.0.5/libraries/SoftwareSerial/SoftwareSerial.cpp so it looks like that:

// .... line 59
static const DELAY_TABLE PROGMEM table[] =
{
    // baud rxcenter rxintra rxstop tx
    { 115200, 1, 17, 17, 12, },
    { 57600, 10, 37, 37, 33, },
    { 38400, 25, 57, 57, 54, },
    { 31250, 31, 70, 70, 68, },
    { 28800, 34, 77, 77, 74, },
    { 19200, 54, 117, 117, 114, },
    { 16064, 66, 140, 140, 137, }, // added baud rate
    { 15625, 68, 144, 144, 141, }, // added baud rate 2
    { 14400, 74, 156, 156, 153, },
    { 9600, 114, 236, 236, 233, },
    { 4800, 233, 474, 474, 471, },
    { 2400, 471, 950, 950, 947, },
    { 1200, 947, 1902, 1902, 1899, },
    { 600, 1902, 3804, 3804, 3800, },
    { 300, 3804, 7617, 7617, 7614, },
};
// ....

Here is the link you should follow for more info. I haven’t figured it out by myself. After this modification Arduino happily sent me my precious data, which I observed in serial monitor.

Serial data fas seen on arduino serial monitor

Serial data fas seen on arduino serial monitor

This is the data I collected so far (motorcycle standing on central stand, back wheel revolving, velocity comes from the back wheel, ABS LED blinking).

Ignition ON, engine stopped.

1 0 0 0 30 30
1 0 0 0 30 30
1 0 0 0 30 30
1 0 0 0 30 30
...

Ignition ON, engine started, no throttle (~1200 RPM), no gear (N) i.e. wheel not revolving, cold engine:

1 24 0 0 30 54
1 24 0 0 30 54
1 24 0 0 30 54
1 24 0 0 30 54
...

Ignition ON, engine started, little throttle (more RPM), no gear (N) i.e. wheel not revolving, cold engine:

1 34 0 0 53 87
1 34 0 0 53 87
1 34 0 0 53 87
1 34 0 0 53 87
...

1st gear, no throttle (engine warmed up thus less RPM). About 10 km/h

1 17 0 0 5C 73
1 17 1 0 5C 74
1 17 1 0 5C 74
1 17 0 0 5C 73
1 17 1 0 5C 74
1 18 1 0 60 79
1 18 1 0 60 79
1 18 0 0 60 78
1 18 1 0 60 79
1 17 1 0 60 78
1 17 0 0 60 77
1 17 1 0 60 78
1 17 1 0 60 78
1 17 0 0 60 77
1 17 1 0 60 78
1 17 1 0 60 78
1 17 0 0 60 77
1 18 1 0 60 79
1 18 1 0 60 79
1 18 1 0 60 79
1 18 0 0 60 78
1 18 1 0 60 79
1 18 1 0 60 79
1 18 0 0 60 78
1 18 1 0 60 79
1 18 1 0 60 79
1 17 0 0 60 77

23 km/h (6th gear)

1 17 2 0 68 81
1 17 2 0 69 82
1 17 2 0 69 82
1 17 2 0 69 82
1 17 1 0 69 81
1 17 2 0 68 81
1 17 2 0 68 81
1 17 2 0 68 81
1 17 2 0 68 81
1 17 1 0 68 80
1 17 2 0 69 82
1 17 2 0 68 81
1 17 2 0 68 81
1 17 1 0 68 80
1 17 2 0 68 81

40 km/h

1 2B 4 0 70 9F
1 2B 3 0 70 9E
1 2B 3 0 72 A0
1 2B 3 0 72 A0
1 2B 4 0 72 A1
1 2B 3 0 72 A0
1 2C 3 0 72 A1
1 2C 4 0 72 A2
1 2C 3 0 72 A1
1 2C 3 0 72 A1
1 2C 4 0 72 A2
1 2C 3 0 72 A1
1 2C 3 0 72 A1
1 2C 4 0 72 A2
1 2C 3 0 72 A1
1 2C 4 0 72 A2
1 2C 3 0 72 A1
1 2C 3 0 72 A1

60 km/h

1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 41 4 0 74 B9
1 41 6 0 74 BB
1 41 4 0 74 B9
1 40 5 0 74 B9
1 40 5 0 74 B9
1 41 5 0 74 BA
1 40 5 0 74 B9
1 40 5 0 74 B9

80 km/h

1 56 7 0 77 D4
1 55 6 0 77 D2
1 56 7 0 77 D4
1 55 6 0 77 D2
1 56 7 0 77 D4
1 56 6 0 77 D3
1 56 7 0 77 D4
1 56 6 0 77 D3
1 56 7 0 77 D4
1 56 6 0 77 D3
1 56 7 0 77 D4
1 56 7 0 77 D4
1 56 6 0 77 D3
1 56 7 0 77 D4
1 56 7 0 77 D4
1 56 6 0 77 D3

Future

Due to uncertainty Raspberry PI serial communication I plan to make custom PCB with AVR in form of shield to Raspberry Pi. It will have two purposes. First it will acquire the data in some similar manner as depicted above, and will pass it to Pi by I2C or SPI (probably with velocity in some more usable form), among with some other data such as brakes and turn-lights, an maybe even outdoor temperature (I always wanted to have this information, especially in spring and autumn). And secondly it will drive some relay to cut the power to Pi which draws 2W of power even when shut down. Of course it also should send some signal first to Raspberry to shut it down correctly. If someone know something more about Pi’s UART and custom baud rates in particular, please let me know. Maybe there is a way to read K-line directly without AVR.

 

Old post about canon remote

So I came up with an idea of Cannon DSLR remote control. They are relatively cheap to buy on ebay, or other local online auction sites like allegro.pl here in Poland. But I wanted to build something by my self. As a complete amateur I wanted to make something small, and simple, thus DIY IR remote control for my camera was born. The protocol was reverse engineered by some smart people over the internet, so all I needed to do was to design the PCB, solder the stuff together, write a program and flash it. Below are the links:

My design is based on ATtiny2131a which in my opinion is a little too powerful, but first tests with ATtiny13 revealed some issues with internal oscillator. The timing of the clock signal is crucial when generating a carrier wave. Carrier should have 32.6kHz frequency for the best results. Deviations from this frequency has significant impact on IR operational range. Without an oscilloscope I was unable calibrate internal oscillator correctly, thus I’ve chosen chip with an external one (in fact the clock frequency was the most difficult issue I’ve got and I spent most of the time dealing with it). After soldering 4MHz quartz into place, at first I set it up for divide by 8 prescaler operation thus giving me 0.5MHz clock signal, but it also failed. I really don’t know why, but it seems, that prescaler is somehow unstalble, or has some sort of overhead (I’m an amateur after all. If someone could explain it, I would be grateful). Finally after setting the chip to operate at full speed (prescaler turned down) I was able to trigger the shutter, but range of operation was still small (circa 3m / 10ft).

I’ve chosen not to drive my IR led directly from AVR pins (like guy in aforementioned links did), but rather use a transistor used as a switch. I assume in that way I’m able to drive bigger current for the IR diode making it to produce stronger flashes. Next to the IR diode you can see a status diode, which indicates to the user, that device is sending a signal. Also I wanted it to be fully reprogrammable thus the connector on the board is present.

The last thing I did was the power down mode. Circuit, when turned on draws ~5mA even, when idling, so I presume the CR2032 battery would not last for long. For prolonging battery life I turn the AVR into power down mode after 3 seconds, and I wake it up via ping change interrupt when button is pressed. AVR in power down mode draws no current at all according to my multimeter (which simply is not precise enough to detect it), but according to the AVR specs it should take about 0.1uA.

Total cost was something about $5-7, with AVR and casing being the most expensive parts.

Below you can find a link to archive with all the necessary files to duplicate my design. Archive include Eagle files (both PCB and schematic), avr-gcc source files with CMake scripts and ATtiny2131a binary). The code was developed in Eclipse.

  • LINK

Feel free to post your thoughts on this design. I would greatly appreciate your comments on what I’ve could do better, or what I’ve done worng (although the device works well, but there always is room for improvement). For example I would be glad to hear how to extend the range which, as I mentioned, is only about 3m / 10ft.

Links

tac

Tac command outputs file in reversed order (thus its name – it’s “cat” but backwards). And sponge is a pure magical one which acts as some kind of buffer which gets all the output and then puts it into whatever file in one go, completely removing its contents (this is how I get it – may be wrong). So one could output modified contents of a file to this file itself as so:

tac diamond.dat | sponge diamond.dat

 

Bloki

Mamy na Jelonkach dwie piwnice. Do dziś był w nich potworny burdel, który posprzątałem i jeśli jeszcze nie zauważyliście, to właśnie się tym chwalę. Strasznie mnie to zmęczyło i zgłodniałem też porządnie, więc poszedłem do chińczyka hał-hała nieopodal, żeby wziąć coś na wynos. I tak sobie siedziałem na zewnątrz, pogoda ładna i tak dalej i jakiś taki dobry humor mi się zrobił. Siedziałem, patrzyłem w dal, gdzie wznoszą się ściany bloków po drugiej stronie ulicy, a nad nimi latają jaskółki. I pomyślałem sobie tak. Gdybym nie miał zupełnie nic i w ogóle nigdy bym niczego nie posiadał, to także nie znał bym takiego pojęcia jak sprzątanie. No bo co może wiedzieć o sprzątaniu ktoś, kto nic nie ma. Wyobraźmy sobie jakiegoś troglodytę, który jedyne co może posprzątać, to liście na trawniku, albo gałązki pod drzewem, ale ponieważ ani trawnik, ani drzewno nie należą do niego (bo nic do niego nie należy), to po co ma to robić. I z jednej strony szczęśliwy jest taki troglodyta, bo troglodyci są znani ze wstrętu do wiosennych porządków, ale z drugiej strony, kiedy przyjdzie na niego czas, i troglodyta zdechnie, to żadna rzecz, żaden kot w pustym mieszkaniu po nim nie pozostanie. Kiedy ja zdechnę, to zostanie po mnie mnóstwo niepotrzebnych śmieci, i ktoś może sobie coś weźmie i pomyśli o mnie chociaż. A jak cała ludzkość nagle zdechnie, to zostaną po niej te bloki, puste mieszkania, puste samochody, puste resztki czegoś, co kiedyś było cywilizacją. No i to w zasadzie to wszystko co wtedy wymyśliłem, bo żona hał-hała przyniosła mi w międzyczasie żarcie na zewnątrz i wróciłem do domu.

Przyjaciele w Radomiu

    Kiedy się ma nową rzecz, na którą się czekało długo tak jak ja, to się tą rzecz oczywiście chce przetestować, nacieszyć, pobawić. Nie inaczej było w przypadku motocykla, tej piekielnej maszyny, którą nabyłem niedawno. Nacieszenie się nowym pojazdem polega na jeżdżeniu w tą i z powrotem, czasem w jakimś celu, a czasem bez. Kiedy już objeździłem całą rodzinę, pochwaliłem się krewnym i znajomym królika i o mało co nie przeziębiłem, powstał dylemat gdzie w następnej kolejności pojechać. Kiedy motocykla nie miałem, wyobrażałem sobie go jako remedium na wszystkie swoje niepowodzenia, sinusoidalne skoki humoru, czasem też całkiem spore dołki. Wyobrażałem sobie siebie jadącego sobie w dal i pogrążonego w swoich myślach, sam na sam ze sobą, odciętego od wszystkiego. Stety lub niestety myliłem się. Okazuje się bowiem, że jadąc na motocyklu człowiek jest (musi być) dwa, lub jeszcze więcej razy bardziej skupiony na tym co się dookoła niego dzieje niż w samochodzie. Tak więc jadąc na moto miałem teraz zupełną pustkę w głowie i jakkolwiek czerpiąc wielką przyjemność z podróży, moja głowa była jednak zaprzątnięta raczej tym, żeby nie dać się zabić, niż rozmyślaniami o sensie istnienia, czy jakichś tam innych pierdół. A Jechałem do Radomia Aleją Krakowską, czy trasą krakowską, jak zwał tak zwał. Radom to paskudne miasto, które pamiętam z dzieciństwa, ponieważ spędzałem tam wakacje u babci Janki i przez babcię, oraz jego paskudność odczuwam do niego spory sentyment (nie taki jak do P-na, ale zawsze). Uzbrojony w karteczkę z numerami tras (wracałem przez Kozienice) i listą miejsc do odwiedzenia (3 pozycje, ale mam słabą pamięć, stąd lista) pędziłem sobie wesoło na południe. Udało mi się tam dojechać w około godzinę, co jednak nie było zbyt rozsądne, jak potem uświadomiły mi żona i matka, no ale cóż.

    Nie jest tak, jak twierdzą niektórzy, że motocykl jest magnesem na płeć piękną i wystarczy się polansować po mieście, żeby od razu jakiś plecak podwieźć, a potem stłamsić. To znaczy może to jest prawda, ale mnie przynajmniej jest to zupełnie obce i to dobrze. Jakkolwiek laski nie kleją się do mojej maszyny, to jednak zupełnie inaczej sprawa się ma z małymi dziećmi. Mam go już miesiąc, a rozmawiałem już chyba z pięciorgiem dzieci w wieku przedszkolnym, o dzieciach gapiących się z mniej dostępnych miejsc nie wspominając. Jeden chłopiec na przykład, lat mniej więcej na oko 2 (bo ledwo mówił) zachęcany przez swojego tatę wymieniał mi wszystkie marki motocykli jakie znał. Trzeba mu było tylko trochę pomagać z pierwszą sylabą, a potem sobie seplenił. Zuch. Inny chciał się gramolić na motocykl, ale babcia na niego nawrzeszczała, jeszcze inne dzieci po prostu przechodząc ciągnęły za klamki, manetki i inne wystające rzeczy. Zupełnie mi to nie przeszkadza.

    Brak mi niestety jeszcze doświadczenia i w Radomiu przydarzył mi się pierwszy paciak. Paciak to (chyba fachowe) określenie na wywrotkę “parkingową” kiedy to toczymy się na motocyklu w celu właśnie wspomnianego parkowania, lub w jakimś innym, dość, że prędkość mamy wtedy prawie żadną. W takich momentach motocykl jest bardzo niestabilny, bowiem brak sił żyroskopowych pochodzących od kręcących się kół (kręcą się wolno, więc siła mała i słabo nas/mnie stabilizuje). I tak samo było w moim przypadku. Zauważyłem bowiem na ulicy Niedziałkowskiego w Radomiu grupę młodzieży gimnazjalnej w sile około dziesięciu na tle wielkiej swastyki wysprejowanej na murze. Szybko się oddaliłem, ale jednak ciekawość pozostała i dokopawszy się do głęboko skrywanych pokładów odwagi postanowiłem do nich wrócić, żeby zrobić im zdjęcie. Niestety nie było ich, ja zacząłem zawracać, samochód się zjawił znikąd, ja po hamulcach i pac. Wygrzebałem się spod motocykla, łażę dookoła i patrzę co by tu zrobić (wyłączył się sam). Próbuję podnieść przodem, nie daję rady. Próbuję inaczej, też nic. Wtem słyszę ożywioną rozmowę. To właśnie owi chłopcy, których szukałem wymieniali się uwagami na temat przyczyn mojego nieszczęścia. Konkluzja brzmiała : “typ się przestraszył”. No i dużo mieli racji, moja gwałtowna reakcja była skutkiem nagłego pojawienia się zagrożenia, którego się nie spodziewałem (to był jakiś stary golf). Szli w moją stronę i zastanawiali się na głos, czy mi pomóc, czy nie. Szybko (na szczęście) doszli do porozumienia, że jednak tak (“pomożemy typowi”) i jeszcze będąc w pół drogi, uspokoili mnie (cały czas szarpiącego się z kawałem żelastwa) w te słowy : “zostaw to, bo się zesrasz!”.

    Zdjęcia już im nie zrobiłem, ale wdzięczny jestem im do dzisiaj, bo strasznie ciężkie to cholerstwo. Oczywiście nie miałem gmoli ani kraszpadów, a motocykl był nowy, więc sami sobie możecie dalej dopowiedzieć co i jak. Na szczęście straty są znikome, a na gmole i kraszpady ciągle zbieram. Pozdrawiam chłopaków z Radomia.