Video-Related Notes

Documentation about the VPU in the JZ4780 programming manual is limited, referring to related documentation which is not publicly available.

These are some notes from studying the Ingenic MPlayer code and other resources including the paper "A Hybrid Scheme Based on Pipelining and Multitasking in Mobile Application Processors for Advanced Video Coding".

Memory Regions

From jz47_vae_map.c and the manual:

Offsets indicated for each VPU peripheral region in the following sections are specified relative to the VPU peripheral base (0x13200000).

CPM Registers

The CPM (clock and power management) registers provide more generally applicable functionality. Registers of pertinence to the VPU are described here.

CPM_CLKGR0 (clock gate register #0) is at offset 0x20.

CPM_VPU_SWRST (soft reset and bus control) is at offset 0xC4.

CPM_CPSPR (scratch pad) at offset 0x34.

CPM_LPCR (low power control) is at offset 0x04.

CPM_OPCR (oscillator and power control) is at offset 0x24.

Reset Operation

1. Set CPM_VPU_STP.
2. Wait for CPM_CPU_ACK to become set.
3. Set CPM_VPU_SR and clear CPM_VPU_STP.
4. Clear CPM_VPU_SR and CPM_VPU_STP.

VPU/SCH Registers

Some registers in this region are defined by libjzcommon/t_vputlb.h. They are given with a VPU prefix above.

Some additional registers appear to exist, defined by macros in jzm_vpu.h:

SCH_GLBC

The register is set to GLBC_HIAXI before the ACFG fields are set in VDMA_TASKRG.

The definitions in libjzcommon/t_vputlb.h do not agree with the above. Instead, the following fields are defined:

The SET_VPU_GLBC macro employs these definitions and is used, whereas the only active definition from the other set is GLBC_HIAXI, which would correspond to GLBC_EPRI as 0b10.

SCH_SLDx

The definitions in libjzcommon/t_vputlb.h do not agree completely with the above. Instead, the following fields are defined:

Page sizes (PSIZE) are 4M (0), 8M (1), 16M (2), 32M (3).

SCH_TLBC

TLB virtual address match register, possibly comparable to EntryHi in the MIPS architecture TLB.

SCH_TLBV

SCH_SCHC

Scheduler channel control.

SCH_SCHEx

From t_vputlb.h:

SCH_BND/SCH_SCHCS

Scheduler group control?

The HID fields employ values corresponding to hardware units. Files such as libh264/jzm_vpu.h provide definitions of the appropriate values as follows:

Alternative symbols are defined in libjzcommon/t_vputlb.h.

These values are provided to macros in order to populate the appropriate channel field in SCH_BND. The GEN_VDMA_ACFG macro employed by files such as libmpeg2/soc/jzm_mpeg2_dec.c may feature such values as in the following example:

GEN_VDMA_ACFG(chn, REG_SCH_BND, 0, (SCH_CH3_HID(HID_DBLK) |
                                    SCH_CH2_HID(HID_VMAU) |
                                    SCH_CH1_HID(HID_MCE) |
                                    SCH_DEPTH(MPEG2_FIFO_DEP)));

SCH_SCHCS

The operation in t_vputlb.h, employing the name SCH_SCHCS, defines the register differently:

However, this operation does not appear to be used.

VDMA Registers

These are defined in jzm_vpu.h as follows:

However, these definitions are not referenced elsewhere, and they might therefore not be valid or relevant. Nevertheless, they are provided here for reference.

Code such as libmpeg2/slice.c does appear to use VDMA_TASKRG and VDMA_TASKST. Meanwhile, the 2D DMA mechanism employs offsets 0x10000 and 0x10004 differently for GP0, along with the corresponding offsets for GP1 and GP2.

VDMA_TASKRG

VDMA_TASKST

Scheduled task status? The bits appear to correspond to those in VDMA_TASKRG.

EFE Registers

MCE Registers

MCE stands for Motion Compensation and Estimation.

MCE_CTRL

Defined in...

• libvp8/t_motion_p0.h
• librv9/jzsoc/t_motion.h
• librv9/jzsoc/t_motion_p0.h

The librv9 definitions differ from the libvp8 definitions.

In vp8.c, all fields are set to zero apart from...

PGC = 0xF, CH2EN = 1, PRI = 3, CKGE = 1, CCF = 1, EN = 1

When decoding a frame, the following additional fields appear to be set:

CAE = 1, OFA = 1

In rv9_p0_mc.c, all fields are set to zero apart from...

CAE = 1, PGC = 0xF, CH2EN = 1, PRI = 3, CKGE = 1, OFA = 1, CCF = 1, EN = 1

MCE_CHx_STRD

Defined in libvp8/t_motion_p0.h and other files.

MCE_GEOM

Defined in libvp8/t_motion_p0.h and other files.

VMAU Registers

VMAU apparently stands for Vector Matrix Arithmetic Unit. The JZ4780 manual indicates that it is used for pixel recovery.

DBLK Registers

SDE Registers

SDE might conceivably stand for "stream decoder".

Also informed by libmpeg2/slice.c.

AUX Registers

Message IRQs are cleared by clearing the appropriate MIRQP register.

AUX_CTRL

If NMI_DIS is clear, a NMI condition will reset AUX and start execution at 0xF4000000. Otherwise, if NMI_DIS is set, AUX will continue from the next instruction after a WAIT instruction.

AUX_SPINLK

The LOCK field is written via a mechanism connected to the AUX_SPIN1 and AUX_SPIN2 registers. Those registers retain a value that is committed to LOCK when they are read and if LOCK is zero.

Despite the presence of these spinlock registers, it seems as if some coprocessor #0 registers are used instead. See, for example, libmpeg4/jzsoc/jz4760_dcsc.h. The pertinent registers are as follows:

Here, the SPINLOCK register's LOCK field (3..0) can only be cleared so that the lock can be taken. Writing a value to the corresponding VAL field (3..0) in SPINATOMIC causes LOCK to be updated with the written value if LOCK was already zero.

Despite provision of the above registers, the following registers appear to be used by libmpeg4/jzsoc/jz4760_dcsc.h for spinlock purposes:

Some documented registers are also available for similar purposes:

AUX_SPIN1

AUX_SPIN2

AUX_MIRQP

AUX_MSG

CORE_MIRQP

CORE_MSG

JPGC Registers

VPU Operations

According to jz47_vae_map.c, reset is performed as follows:

Soft Reset

1. CPM_VPU_SWRST has bit 30 (STP) set.
2. CPM_VPU_SWRST bit 29 (ACK) tested repeatedly until it is set.
3. CPM_VPU_SWRST bit 31 (SR) is set and bit 30 (STP) cleared.
4. CPM_VPU_SWRST bit 31 (SR) is cleared and bit 30 (STP) cleared.

According to libjzcommon/t_vputlb.h, the following operations are defined.

Reset

1. Save the first word of each 2048-word page for the six pages of TCSM.
2. CPM_CPSPR is set to zero.
3. CPM_LPCR has bit 30 (PD_VPU, power down module VPU) cleared.
4. CPM_OPCR has bit 29 (MASK_VPU, debugging bit) set. This may be optional.
5. CPM_OPCR bit 28 is tested repeatedly until it becomes set.
6. CPM_OPCR bit 30 (PD_VPU) is set and bit 29 (MASK_VPU) is cleared.
7. CPM_OPCR bit 30 (PD_VPU) is cleared, twice!
8. The first word of each 2048-word page is restored.

GLBC

This only seems to be used in libvp8/vp8.c:

SET_VPU_GLBC(1, 0, 0, 4, 4);

This sets GLBC_TLBE as 1, GLBC_ENGM as 0, GLBC_EPRI as 0, GLBC_DPRI as 4, GLBC_CPRI as 4.

TLB

SET_VPU_TLB(entry, valid, psize, vtag, ptag)

This appears to set the SCH_SLDE0..SCH_SLDE3 registers with validity, page size, virtual tag, physical tag. The tags appear to be the uppermost ten bits of addresses, but in the MPlayer code an identity mapping appears to be used, employing physical addresses as virtual addresses.

Scheduling

SET_VPU_SCHC(sch4_act, sch4_pe1, sch4_pch1,
                       sch4_pe0, sch4_pch0,
             sch3_act, sch3_pe0, sch3_pch0,
             sch2_act, sch2_pe0, sch2_pch0,
             sch1_act, sch1_pe0, sch1_pch0)

This is used in libh264/jzsoc/h264_p1.c and libh264/jzsoc/h264_cavlc_p1.c, with initialisation setting the ACT fields to 0, the PE fields to 1, and the PCH fields to 3, 2, 1, 1 and 0 respectively. All fields are set to 0 at the end or if an error occurs.

Clocks

The "A Hybrid Scheme..." paper indicates that the JZ4770 has its main "J1" core running at 1000 MHz with the AUX core running at 500 MHz.

Memory Layout

The different JZ-series manuals provide the following details about memory layout for the VPU facilities:

SRAM is Scratch RAM. The SRAM appears to differ between the JZ4760 and JZ4780, having 32K in the former at 0x132D0000, but only 16K in the latter. The MPlayer code (for example, libmpeg4/jzsoc/jz4760_tcsm_init.c) suggests a SRAM size of only 7K. However, elsewhere (for example, librv9/jzsoc/rv9_sram.h) the size is at least 16K, and in jz47_vae_map.c it is given as being 0x7000 bytes (28K) which is consistent with what is written in the "A Hybrid Scheme..." paper:

"There are three types of on-chip memories including two tightly coupled shared memories TCSM0 (16 KB) and TCSM1 (48 KB) along with SRAM (scratch RAM 28 KB)."

This is reflected in the jz47_vae_map.c definitions and is presumably definitive. Other files also seem to employ compatible definitions. For example:

• libh264/jzsoc/mem_def_falcon.h
• libh264/jzsoc/eyer_driver.h

Each region has its own DMA channel:

• TCSM0 is accessible by DMA0
• TCSM1 is accessible by DMA1
• SRAM is accessible by DMA2

The AUX core, at least according to the more helpful JZ4760 programming manual, "can only access physical address space". However, it seems that it employs the following fixed memory mappings within TCSM1:

Thus, the main core and the AUX core each address "their own" TCSM via the 0xF4000000 region, with the DMA channels being employed to move data around.

Physical addresses outside the above mappings are accessible via the unmapped kernel mode memory regions:

This is featured in libmpeg4/jzsoc/mpeg4_tcsm0.h.

Code Positioning

Some files define code for the p1_main section, such as libh264/jzsoc/h264_p1.c. A common linker script appears to be used to define the positioning of such code, found in the following locations:

• libjzcommon/jz4760e_p1.ld
• libffmpeg2/jzsoc/ffmpeg2_p1.ld
• libmpeg4/jzsoc/mpeg4_p1.ld

The p1_main section appears at 0xF4000000, which for the AUX core appears to address TCSM1. So, code in this section may be intended for the AUX core and define a dedicated process running on that core.

AUX Operations

Reset

According to libmpeg4/jzsoc/jz4760_dcsc.h, AUX is reset by writing to bit 0 (SW_RST) of AUX_CTRL.

Start

According to libmpeg4/jzsoc/jz4760_dcsc.h, AUX is started as follows:

1. AUX_CTRL is set to 1, initiating a software reset (SW_RST)
2. AUX_CTRL is set to 2, initiating a non-maskable interrupt (SW_NMI)

Since the NMI_DIS field will be clear in both of the above operations, the NMI initiation will cause the AUX core to start executing code at 0xF4000000.

Codecs

H.264

Files: libh264/jzsoc/h264_p1.c, libh264/jzsoc/h264_cavlc_p1.c

P1_START_BASE is defined in libh264/jzsoc/mem_def_falcon.h and included files as...

GP0_DHA_BASE + GP0_DHA_SIZE (TCSM0_1VALUE + 4) + (4 * 4) ((C_TCSM0_BASE + 16) + 4) + (4 * 4)

...which corresponds to 0x132B0000 + 36.

It is set to zero at the start of processing and to AUX_END_VALUE (0x5A5A) at the end of processing, having posted a message word of 1 to AUX_MSG.

MPEG2

mpeg2_parse

File: libmpeg2/decode.c

Resets the VPU using the CPM_VPU_SWRST register. Calls mpeg2_slice.

mpeg2_slice

File: libmpeg2/slice.c

Calls (depending on the code parameter) M2D_SliceInit and M2D_SliceInit_ext.

Initialisation:

1. VDMA_TASKST is set to zero.
2. SDE_STAT is set to zero.
3. SCH_STAT is set to zero.
4. SCH_GLBC is cleared apart from HIAXI which is set.
5. VDMA_TASKRG is set to (VDMA_ACFG_DHA(s->des_pa) | VDMA_ACFG_RUN).

In the above, s->des_pa is the physical address of the allocated frame, defined in decode.c as follows:

mpeg2dec->decoder.vdma_base = (uint8_t *) jz4740_alloc_frame(128, 0x2000);

There is then a loop whose continuation condition is that SCH_STAT bit 0 is clear. In other words, it presumably terminates when SCH_STAT bit 0 is set. Presumably, SCH_STAT bit 0 has a similar role to VDMA_TASKST bit 0, which is VPU_BUSY.

M2D_SliceInit

File: libmpeg2/soc/jzm_mpeg2_dec.c

Sets up the DMA configuration using GEN_VDMA_ACFG.

M2D_SliceInit_ext

File: libmpeg2/soc/jzm_mpeg2_dec.c

Sets up the DMA configuration using GEN_VDMA_ACFG.

2D DMA

The jz4760e_2ddma_hw.h file appears to describe DMA for transferring data between the TCSM and SRAM areas. However, its definitions appear to employ the VDMA region, together with other neighbouring regions, differently to that described in jzm_vpu.h. The JZ4770 manual does, however, provide a description of the descriptor formats.

GPx_DHA

Technically, the whole register represents the address at word (4-byte) resolution.

GPx_DCS

The DCS register employs bit 2 as an apparent completion indicator.

Descriptors

The descriptors employed by the GP0, GP1 and GP2 channels also appear to be different to those employed in the VDMA initialisation:

The transfer size types (TST) can be...

• 0b00 - word
• 0b01 - byte
• 0b10 - half-word
• 0b11 - illegal value

The page size optimisation (FRM) can be...

• 0b00 - unset
• 0b01 - 1K
• 0b10 - 2K
• 0b11 - 4K

The "current node link" (TAG) appears to be 1 at the final descriptor, 0 otherwise.

The file libffmpeg2/jzsoc/ffmpeg2_p1.c provides an example of a transfer from TCSM0 to TCSM1:

set_gp0_dha(TCSM1_PADDR(DDMA_GP0_SET));
*((volatile int *)(DDMA_GP0_SET + TSA))  = TCSM0_PADDR(*tcsm1_fifo_rp);
*((volatile int *)(DDMA_GP0_SET + TDA))  = TCSM1_PADDR(dMB);
*((volatile int *)(DDMA_GP0_SET + STRD)) = GP_STRD(TASK_BUF_LEN, 0,
                                                   TASK_BUF_LEN);
*((volatile int *)(DDMA_GP0_SET + UNIT)) = GP_UNIT(1, TASK_BUF_LEN,
                                                   sizeof(struct MPEG2_MB_DecARGs));
set_gp0_dcs();
poll_gp0_end();

This employs physical addresses for the descriptor, source address and destination address. GP_STRD defines the source and destination strides with no DDR optimisation. GP_UNIT defines the final descriptor with the row width as the same as the stride, with the total transfer being defined by the size of the referenced structure.

VDMA

The macro GEN_VDMA_ACFG in jzm_vpu.h appears to be involved in writing descriptor fields in the following format:

Decoder initialisation such as in the following files employs this macro to set up DMA descriptors:

• libh264/jzm_h264_dec.c
• libmpeg2/soc/jzm_mpeg2_dec.c
• libvc1/soc/jzm_vc1_dec.c

Files

Some files in the MPlayer sources that provide information.

jz47_vae_map.c

Mapping of memory regions. This appears to contain the definitive regions for the JZ4780.

jz47_soc_mem.c

IPU-related memory mapping and allocation.

jzm_vpu.h

VPU register definitions and codec data. libh264/jzm_vpu.h appears to be the most complete of the definitions, with an older version appearing in libmpeg2/soc/jzm_vpu.h and libvc1/soc/jzm_vpu.h.

Note that libjzcommon/t_vputlb.h provides differing definitions but ones that appear to be actually used, whereas those in jzm_vpu.h may not be. Another variant of t_vputlb.h exists in libmpeg4/jzsoc alongside files such as t_motion.h and t_intpid.h.