Cyclone V: uboot-socfpga with SPL/fpga load

jackfrye11 · April 9, 2020, 12:59am

Can anyone here verify that the this new u-boot, which combines support for the secondary preloader and u-boot into one u-boot-with-spl binary, is capable of loading a CycloneV FPGA?

This is a hugely important feature for SoCs and I cannot imagine it was overlooked, but I have been blocked on trying to get this feature to work for months. If there is some bug, I would like to report it or at least how to diagnose the problem.

man27 · April 30, 2020, 11:03pm

I have the same issue. I am also working on trying to solve it. I’ll post here if I find a solution.

thaler · May 1, 2020, 12:59am

I’m using U-Boot v2019.04 for quite a while. There is only a single “u-boot-with-spl.sfp” as output.

jackfrye11 · May 1, 2020, 1:16am

Right. That is fine.
You should be able to build u-boot-with-spl.sfp so that you can get through the SPL, which is working fine as far as I can tell.
Then you should be able to point u-boot to a script that contains commands to interface with the FPGA manager in order to load a CycloneV FPGA image off of the SD Card or out of SPI Flash and program it into the FPGA

fatload mmc 0:1 ${fpgadata} socfpga.rbf;
fpga load 0 ${fpgadata} ${filesize};
bridge enable;

Right now, I can not do this and I do not know anyone on this forum who has. Back at Christmas time, I emailed Tom Rini, who at that time had the latest u-boot commits to socfpga_cyclone5_defconfig, detailing the issues and asking for guidance. He never got back to me, nor has anyone been able to program CycloneV FPGA from u-boot since this new build flow came out.

To date, my best investigation has led me here GSRD v13.1 - Programming FPGA from HPS | Documentation | RocketBoards.org, which details that the -6 error code I receive every time I try to program the FPGA is due to “The FPGA is having timeout from entering init phase or user mode after disable AXI configuration at FPGA Manager”. Perhaps I am configuring the HPS incorrectly in my Quartus build, but I believe I once configured the HPS the same way it is documented in this excellent guide from some former Cornell undergrads https://people.ece.cornell.edu/land/courses/ece5760/DE1_SOC/SoC-FPGA%20Design%20Guide_EPFL.pdf and I was able to program the FPGA from u-boot using the old flow with separate SPL and u-boot

Although I am just a hobbyist on the Intel platform, it seems to me like there is a bug and no one with the expertise to date has been able to offer insight on this forum.

thaler · May 1, 2020, 1:06pm

I’m also loading an RBF via boot script into the FPGA. That works fine, too. Exactly as you said:

# load FPGA config
ext4load mmc 0:2 0x2000000 /boot/socfpga.rbf
fpga load 0 0x2000000 7007204

My U-Boot version is a year old (4/2019) maybe they broke something in the meantime.

You can find the U-Boot version I’m using here:

It definitely works for my DE10-Nano. If it is not working at your site, the problem is probably outside of U-Boot, e.g. broken RBF file or SD card.

jackfrye11 · May 2, 2020, 4:39pm

I am struggling to get anything to appear on my console at boot.

I think it is a problem with my SoC design. Do you have a qpf with a qsys example of the CycloneV setup that you are using that I might see. If it’s proprietary, no worries.

@dhyslop was able to boot into Linux with the current setup as detailed in this thread. Cyclone V: Preloader and Bootloader, the new workflow since SoC EDS Standard version 19.1

It does not say explicitly, but he sent me a DM to confirm his efforts detailed in the 4/19 post were successful

Before I start playing with your u-boot, I would like to ensure the pieces around it are operational by replicating what he has done.

jackfrye11 · May 3, 2020, 1:57am

Ok. I was able to boot into Linux.

@thaler , can you share your boot script and which headers you edited in u-boot to get it to run?

I will start with trying to changes highlighted by this guide.
Per this guide

which is for an older version of u-boot.

I will change u-boot-socfpga/include/configs/socfpga_common.h from

#define CONFIG_EXTRA_ENV_SETTINGS \
	"fdtfile=" CONFIG_DEFAULT_FDT_FILE "\0" \
	"bootm_size=0xa000000\0" \
	"kernel_addr_r="__stringify(CONFIG_SYS_LOAD_ADDR)"\0" \
	"fdt_addr_r=0x02000000\0" \
	"scriptaddr=0x02100000\0" \
	"pxefile_addr_r=0x02200000\0" \
	"ramdisk_addr_r=0x02300000\0" \
	"socfpga_legacy_reset_compat=1\0" \
	BOOTENV

to

#define CONFIG_EXTRA_ENV_SETTINGS
“scriptfile=u-boot.scr” “\0”
“fpgadata=0x2000000” “\0”
“callscript=fatload mmc 0:1 $fpgadata $scriptfile;”
“source $fpgadata” “\0”

I will then change u-boot-socfpga/include/config_distro_bootcmd.h

#define CONFIG_BOOTCOMMAND “run distro_bootcmd”

to

#define CONFIG_BOOTCOMMAND “run callscript”

The guide uses a script that looks like this

echo --- Resetting Env variables ---
# reset environment variables to default
env default -a
echo --- Setting Env variables ---
# Set the kernel image
setenv bootimage zImage;
# address to which the device tree will be loaded
setenv fdtaddr 0x00000100
# Set the devicetree image
setenv fdtimage socfpga.dtb;
# set kernel boot arguments, then boot the kernel
setenv mmcboot 'setenv bootargs mem=1024M console=ttyS0,115200 root=${mmcroot} rw rootwait; bootz
${loadaddr} - ${fdtaddr}';
# load linux kernel image and device tree to memory
setenv mmcload 'mmc rescan; ${mmcloadcmd} mmc 0:${mmcloadpart} ${loadaddr} ${bootimage}; ${mmcloadcmd}
mmc 0:${mmcloadpart} ${fdtaddr} ${fdtimage}'
# command to be executed to read from sdcard
setenv mmcloadcmd fatload
# sdcard fat32 partition number
setenv mmcloadpart 1
# sdcard ext3 identifier
setenv mmcroot /dev/mmcblk0p2
# standard input/output
setenv stderr serial
setenv stdin serial
setenv stdout serial
# save environment to sdcard (not needed, but useful to avoid CRC errors on a new sdcard)
saveenv
################################################################################
echo --- Programming FPGA ---
# load rbf from FAT partition into memory
fatload mmc 0:1 ${fpgadata} socfpga.rbf;
# program FPGA
fpga load 0 ${fpgadata} ${filesize};
# enable HPS-to-FPGA, FPGA-to-HPS, LWHPS-to-FPGA bridges
bridge enable;
################################################################################
echo --- Booting Linux ---
# load linux kernel image and device tree to memory
run mmcload;
# set kernel boot arguments, then boot the kernel
run mmcboot;

Will post on any progress.
If you have something I can copy, that would be very helpful.

jackfrye11 · May 3, 2020, 2:29am

@thaler
I got the same thing I have gotten before we started our discussion in this thread.

U-Boot SPL 2019.04-07341-gbc38a09daa-dirty (May 02 2020 - 22:08:02 -0400)
Trying to boot from MMC1

U-Boot 2019.04-07341-gbc38a09daa-dirty (May 02 2020 - 22:08:02 -0400)

CPU: Altera SoCFPGA Platform
FPGA: Altera Cyclone V, SE/A5 or SX/C5 or ST/D5, version 0x0
BOOT: SD/MMC Internal Transceiver (3.0V)
Watchdog enabled
DRAM: 1 GiB
MMC: dwmmc0@ff704000: 0
Loading Environment from MMC… OK
In: serial
Out: serial
Err: serial
Model: Altera SOCFPGA Cyclone V SoC Development Kit
Net: eth-1: ethernet@ff702000
Hit any key to stop autoboot: 0
1787 bytes read in 4 ms (435.5 KiB/s)

Executing script at 02000000

— Resetting Env variables —

Resetting to default environment

— Setting Env variables —
Saving Environment to MMC… Writing to MMC(0)… OK
— Programming FPGA —
7007204 bytes read in 372 ms (18 MiB/s)
Command ‘load’ failed: Error -6
— Booting Linux —
4978264 bytes read in 267 ms (17.8 MiB/s)
fatload - load binary file from a dos filesystem

Usage:
fatload [<dev[:part]> [ [ [bytes [pos]]]]]
- Load binary file ‘filename’ from ‘dev’ on ‘interface’
to address ‘addr’ from dos filesystem.
‘pos’ gives the file position to start loading from.
If ‘pos’ is omitted, 0 is used. ‘pos’ requires ‘bytes’.
‘bytes’ gives the size to load. If ‘bytes’ is 0 or omitted,
the load stops on end of file.
If either ‘pos’ or ‘bytes’ are not aligned to
ARCH_DMA_MINALIGN then a misaligned buffer warning will
be printed and performance will suffer for the load.
mmc - MMC sub system

Usage:
mmc info - display info of the current MMC device
mmc read addr blk# cnt
mmc write addr blk# cnt
mmc erase blk# cnt
mmc rescan
mmc part - lists available partition on current mmc device
mmc dev [dev] [part] - show or set current mmc device [partition]
mmc list - lists available devices
mmc hwpartition [args…] - does hardware partitioning
arguments (sizes in 512-byte blocks):
[user [enh start cnt] [wrrel {on|off}]] - sets user data area attributes
[gp1|gp2|gp3|gp4 cnt [enh] [wrrel {on|off}]] - general purpose partition
[check|set|complete] - mode, complete set partitioning completed
WARNING: Partitioning is a write-once setting once it is set to complete.
Power cycling is required to initialize partitions after set to complete.
mmc setdsr - set DSR register value

FDT and ATAGS support not compiled in - hanging

ERROR ### Please RESET the board

I am using the branch from your repository labeled socfpga_v2019.04

My guess is that it is either my script (although I cannot understand how my commands are wrong for programming the FPGA) or maybe my Quartus isn’t right.

What version of Quartus are you using?
When I look at my version in Help->About Quartus Prime, it says

Quartus Prime Version 19.1.0 Build 670 09/22/2019 SJ Lite Edition

thaler · May 4, 2020, 6:12pm

My boot script looks like this:

# compile with: mkimage -T script -C none -n "boot" -d boot.scr boot.img

setenv fdtimage socfpga_de0_nano_soc.dtb
setenv bootimage zImage
setenv root /dev/mmcblk0p2

setenv fpga 0
setenv fdtaddr 0x1000
setenv kerneladdr 0x200000
setenv fpgadata 0x2000000
setenv bootargs console=ttyS0,115200 debug ignore_loglevel earlyprintk=ttyS0,115200 break=y root=${root} rootwait rw ${bootargs_extra} ${runlevel}

# load FPGA config
ext4load mmc 0:2 0x2000000 /boot/socfpga.rbf
fpga load 0 0x2000000 7007204

# load kernel and DT
ext4load mmc 0:2 ${kerneladdr} /boot/${bootimage}
ext4load mmc 0:2 ${fdtaddr} /boot/${fdtimage}

# setup bridges
bridge enable 0xffffffff

# boot kernel
bootz ${kerneladdr} - ${fdtaddr}

jackfrye11 · May 4, 2020, 6:28pm

For the last argument to
fpga load

How do you get the size of the image? Seems like a magic number. What if my image is a different size.

When I get a chance, I’ll give your script a shot. Thanks so much.

thaler · May 4, 2020, 6:29pm

It is not neccessary to have the FPGA programmed to boot the kernel. It may lead to problems later in the boot process, if you have any devices in your DeviceTree that rely on a programmed FPGA.
You should skip this step for now.

It seems that fatload fails. From what I see, loadaddr is empty in the script. Recheck the parameters used in the mmcload variable. I think you’re really close to have a booting kernel

jackfrye11 · May 4, 2020, 6:50pm

@thaler
I know I can get into the kernel. I have done this with the default config in the u-boot that you shared earlier. I think the arguments for the script I was using were not right for booting the kernel, even if they were correct for programming the FPGA.

I really want to write platform drivers and load them at boot so everything in the kernel is initialized at login. (I like automating things.) To do this, I need the fabric configured before I start probing the AXI busand the kernel enumerates devices. That is my underlying motivation for this whole thing, even though I know it is not necessary.

My question about the image size still stands. How do you get that number?
I also wondered what the bitmask argument to the bridge enable command is. That is another difference between our scripts

thaler · May 4, 2020, 11:25pm

It is the size of the RBF in bytes. You also get this number returned by the ext4load (or fatload) in the $filesize variable.

[00:54:17:564] 7007204 bytes read in 383 ms (17.4 MiB/s)

It tells U-Boot which of the three bridges to enable. Since there are only three bridges, I guess 0x7 is enough to enable all three. I’m not sure about their order. You can check in source code.

jackfrye11 · May 5, 2020, 1:26am

@thaler
I ran your script. This time I boot into the kernel fine, but I still can’t load the fpga

U-Boot 2019.04-07341-gbc38a09daa-dirty (May 02 2020 - 22:08:02 -0400)

CPU: Altera SoCFPGA Platform
FPGA: Altera Cyclone V, SE/A5 or SX/C5 or ST/D5, version 0x0
BOOT: SD/MMC Internal Transceiver (3.0V)
Watchdog enabled
DRAM: 1 GiB
MMC: dwmmc0@ff704000: 0
Loading Environment from MMC… *** Warning - bad CRC, using default environment

In: serial
Out: serial
Err: serial
Model: Altera SOCFPGA Cyclone V SoC Development Kit
Net: eth-1: ethernet@ff702000
Hit any key to stop autoboot: 0
707 bytes read in 3 ms (229.5 KiB/s)

Executing script at 02000000

7007204 bytes read in 385 ms (17.4 MiB/s)
Command ‘load’ failed: Error -6
4978264 bytes read in 275 ms (17.3 MiB/s)
26612 bytes read in 5 ms (5.1 MiB/s)

Flattened Device Tree blob at 00001000

Booting using the fdt blob at 0x001000
Loading Device Tree to 03ff6000, end 03fff7f3 … OK

Starting kernel …

What other variables could it be? Is there any chance you could send me a verified .rbf to try. Perhaps it’s my Intel Quartus tools?

jackfrye11 · May 5, 2020, 2:37am

This is going to sound REALLY bad
I started looking through the DE1-SoC user guide and I saw this table
de1_table

I had the switches in the configuration 01010.
I switched it to 00000, and the programming operation in u-boot worked.
@thaler , sorry for going through all this with you. Hopefully you can get a laugh out of this. I have been working on this for months and it was a board DIP switch.

@man27 where are you struggling? Perhaps I can assist you.

Thanks.

man27 · May 5, 2020, 9:15pm

@jackfrye11
I am struggling with getting FPGA load to load the FPGA binary image .rbf .

I get the same error -6 message.

My board is DE0-nano.

Unfortunately, the switches are in the correct setting which is ON-ON-ON-ON-ON-X

Which u-boot branch did you use? 2019.10 or 2019.04?

Edit:
Would this work with Quartus version 18.0??

jackfrye11 · May 5, 2020, 11:15pm

I used Quartus Lite 19.1
I used @thaler’s u-boot repo

I checked out branch socfpga_v2019.04

I used the exact script from post 9 that @thaler posted with the changes in u-boot that I made explicitly in post 7. I compiled the script not as the name he compiled it to (which was boot.img) and made changes to the file names in the script to match my files, and named my compiled script u-boot.scr to match the changes I made in the u-boot environment variables.

I included

the compiled device tree (in my case socfpga_cyclone5_socdk.dtb)
the kernel boot image (zImage)
the rbf (in my case I called it hps_only.rbf)

in the rootfs in the /boot/ directory as thalers script would suggest.

I used the make_image.py script documented in https://rocketboards.org/foswiki/Documentation/BuildingBootloader

and ran it with this command

sudo ./make_sdimage.py -f \
-P u-boot-with-spl.sfp,num=3,format=raw,size=10M,type=A2  \
-P sdfs/*,num=1,format=vfat,size=100M \
-P rootfs/*,num=2,format=ext3,size=300M \
-s 512M \
-n sdcard_cv.img

Where the ./sdfs had the compiled u-boot.scr
and ./rootfs had the root file system with the extra files added in the /boot directory as described above.

Copied the sdcard_cv.img onto the sdcard using Balena Etcher

===========================================================
If you can get into Linux, I would suggest you play with the fpga manger. I was able to read the status and state files for fpga0. The state was “powered off”. That alerted me that there was something wrong at the board level. That’s when I rechecked the electrical connections and reviewed the DIP switch settings. If your fpga is powered off, but you are sure your DIP switches and other circuit settings are right, maybe your board has an electrical problem.

Hope this helps.

man27 · May 6, 2020, 8:09pm

Thanks for the summary. I will try it out and post the results.

Edit:
I was not able to load FPGA image working for Terasic DE0-nano board with Thaler’s u-boot branch.

However, I strangely was able to get a DE10-nano board to work with the newer u-boot 2019.10 and Quartus 18.0 lite using the new bootloader guide and u-boot.scr to boot kernel and load FPGA image.

I believe my DE0-nano board is faulty or not supported by u-boot since changing the switches did not make any difference.

Thank you @jackfrye11for posting this thread.

man27 · May 13, 2020, 9:25pm

It turns out that I made a mistake. I had Quartus building the firmware image .rbf for the wrong device. I changed it to DE0-nano and the fpga image successfully loaded!

kudos3110 · May 25, 2021, 7:05pm

Hi @jackfrye11 and @thaler,

Sorry for digging this up. However, I do have some confusion regarding the u-boot.scr file. As I followed your instruction, I do make my fpga board (DE10 Standard) to load up the .rbf file. I tested it with the original u-boot.script in which the needed files are saved in mmc 0:1 (picture 1) and this did not went well, next, I tried thaler’s script and put all the files into mmc 0:2 (picture 2) and it works. So I did a little test with the first version, and I realize that the “fatload” function did not work at all. Comparing the “fatload” and “ext4load”, they are similar in syntax, and my .rbf was obviously recognized by the system. So I wonder if you guys have looked into this and somehow make the “fatload” work, as saving the required files in the FAT32 region will save a lot of time with the upgrading of the system.

Also, what do you guys think about completely removing the u-boot.scr and put all the booting instruction in the extlinux.conf file?

Best regards,

Viet