Problem with fpga2hps Bridge

bmorcos · May 25, 2017, 9:21pm

I was reading through this post /sys/class/fpga-bridge empty? and I have a slightly different issue… but maybe related? I am using a Cyclone V Dev Kit with Angstrom 2015.12 and kernel 4.1.33-ltsi. Also, I’m using the opencl.rbf instead of the standard soc_system.rbf shipped with the GSRD as I am working towards using opencl.

I can successfully insert the aclsoc_drv
But then I try to run a program, how about the vector_add example I am missing the required fpga_bridge files

Initializing OpenCL
Platform: Intel(R) FPGA SDK for OpenCL(TM)
Using 1 device(s)
  c5soc : Cyclone V SoC Development Kit
Using AOCX: vector_add.aocx
Reprogramming device [0] with handle 1
sh: /sys/class/fpga-bridge/fpga2hps/enable: No such file or directory
sh: /sys/class/fpga-bridge/hps2fpga/enable: No such file or directory
sh: /sys/class/fpga-bridge/lwhps2fpga/enable: No such file or directory
Couldn't open FPGA status from /sys/class/fpga/fpga0/status!
sh: /sys/class/fpga-bridge/fpga2hps/enable: No such file or directory
sh: /sys/class/fpga-bridge/hps2fpga/enable: No such file or directory
sh: /sys/class/fpga-bridge/lwhps2fpga/enable: No such file or directory
mmd program_device:  Board reprogram failed

MMD FATAL: acl_pcie.cpp:47: can't find handle -1 -- aborting
host_intel: acl_pcie.cpp:47: ACL_PCIE_DEVICE* get_pcie_device(int): Assertion `0' failed.

It looks like the bridges are registered properly on boot:

root@cyclone5:~# dmesg | grep fpga
[    2.133638] fpga_manager fpga0: Altera SOCFPGA FPGA Manager registered
[    2.140708] altera_hps2fpga_bridge sopc@0:fpgabridge@0: fpga bridge [hps2fpga] registered
[    2.149080] altera_hps2fpga_bridge sopc@0:fpgabridge@1: fpga bridge [lwhps2fpga] registered
[    2.180212] altera_hps2fpga_bridge sopc@0:fpgabridge@2: fpga bridge [fpga2hps] registered
[    2.188772] altera_fpga2sdram_bridge sopc@0:fpgabridge@3: fpga bridge [fpga2sdram] registered
[    2.197279] altera_fpga2sdram_bridge sopc@0:fpgabridge@3: driver initialized with handoff 000001ff

In my case the fpga_bridge folder is not empty and the bridges all have an appropriate entry for name:

root@cyclone5:~# ls /sys/class/fpga_bridge
br0  br1  br2  br3
root@cyclone5:~# ls /sys/class/fpga_bridge/br0
device     name       of_node    power      state      subsystem  uevent

Maybe these just need to be renamed? but I cannot just rename them live (not permitted), probably in a device table somewhere? Any ideas?

AdeliWamg · May 30, 2017, 9:49am

hi,

Firstly, pls check the bridge status(0 means disabled and 1 means enabled

) using the command in Linux console as below,

cat /sys/class/fpga-bridge/*/enable

if disabled, then you can try to type the commands below in uboot console to enable the bridges

fpgabr 1
run mmcload (boot from ssd)
run mmcboot (boot from ssd)

or

run bridge_enable_handoff
run mmcload (boot from ssd)
run mmcboot (boot from ssd)

B.R.
Adeli Wang

bmorcos · May 30, 2017, 12:43pm

Hi Adeli,

Thank you for the reply. The naming convention in this build is different that what you described, but all 4 bridges are enabled:

root@cyclone5:~# cat /sys/class/fpga_bridge/*/state
enabled
enabled
enabled
enabled

I am still thinking that these just need to be renamed appropriately (eg. fpga2hps) but I am not sure where to do that exactly as I cannot simply rename them while logged into the board,

AdeliWamg · May 31, 2017, 2:19am

it seems vary from kernel release. How likely to try another release ?

AdeliWamg · May 31, 2017, 2:30am

BTW， there are a couple of CV SoC based opencl design examples available at

https://www.altera.com/products/design-software/embedded-software-developers/opencl/developer-zone.html

BriarPatch · August 16, 2017, 3:29pm

Hello bmorcos,
I’ve hit the same problem trying to configure a DE10_standard board with kernel 4.7.0 and using OpenCL v17.
the names for the fpga bridge have gone from fpga-bridge to fpga_bridge and then named br0 -br3 under there.
Did you find a solution?

It would be useful if somewhere it stated which versions of kernels would run with the different version numbers of OpenCL. Has anyone seen a document that states this?

Best wishes,

J

bmorcos · August 16, 2017, 8:11pm

Unfortunately I haven’t found a solution and I haven’t looked much at this recently.

I got a little feedback from Intel saying that the new names (brx) are due to a driver rewrite, so my best guess would be a version mismatch somewhere in the BSP/kernel/driver/device tree or something. I really have no idea honestly.

I would be interested to hear if you make any progress. Have you tried compiling using the old kernel that is shipped with the design (I think 3.10 or 3.13 if I recall)? That was next on my list before I put it on the back burner.

BriarPatch · August 17, 2017, 10:26am

Hello bmorcos,

Thanks for the reply. As far as I can work out it seems that the Linux V4 Kernels introduced the name change for the fpga bridge.

In the SoC porting guide Intel-Altera recommend socfpga-3-13-rel14.0 however I couldn’t get that to build.

I’m using a Terasic DE10-Standard and the image shipped with that is a 3.10 31-ltsi (on Angstrom v2014.12) for the V16.1 implementation of OpenCL.

I’ll see if I can get V17 running on that.

Best wishes,

J

bmorcos · August 17, 2017, 1:00pm

To compile the older kernel you will need an older toolchain also, did you try an older version of gcc for the build?

JanKonecny · October 24, 2017, 6:35am

Hello,
could I ask You about any progress in problem of enabling/disabling bridges? I have been searching for the answer and trying myself various manners practically for quite a long time but I am still not able to controll state of bridges. It seems not to be any official information about the change of behaviour of bridges.
I am using Linux kernel 4.1.22-ltsi (even if the version 4.1.33-ltsi should be supported by Intel/Altera now).
I thanks for any answer in advance.
Best wishes, Jan Konecny.

JanKonecny · April 30, 2018, 2:37pm

Hello,
I have found out the answer myself.
The proper way to reprogramme FPGA from running operating system is using DeviceTree overlays. The driver of Altera FPGA manager disnables and enables all necessary bridges itself and programmes FPGA with Raw Binary File (RBF) according to information written in DeviceTree overaly file (.dtbo).
I means that Yours Kernel have to be able to use DeviceTree overlays and support ConfigFS filesystem.
You can read more in Documentation in sources of Linux kernel code, especially here:
https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.9.78-ltsi/Documentation/devicetree,
https://github.com/altera-opensource/linux-socfpga/blob/socfpga-4.9.78-ltsi/Documentation/devicetree/bindings/fpga/fpga-region.txt.
DTS:

/*

This devicetree is generated by sopc2dts version 17.0 [9b3346002ac555f36b80b1bc56dad1cb86298234] on Thu Mar 01 11:38:46 CET 2018

Sopc2dts is written by Walter Goossens waltergoossens@home.nl

in cooperation with the nios2 community nios2-dev@lists.rocketboards.org
*/
/dts-v1/;

/ {
model = “ALTR,soc_system”;
compatible = “ALTR,soc_system”;
#address-cells = <1>;
#size-cells = <1>;

aliases {
ethernet0 = “/sopc@0/ethernet@0xff702000”;
}; //end aliases

cpus {
#address-cells = <1>;
#size-cells = <0>;
  hps_0_arm_a9_0: cpu@0x0 {
  	device_type = "cpu";
  	compatible = "arm,cortex-a9-17.0", "arm,cortex-a9";
  	reg = <0x00000000>;
  	next-level-cache = <&hps_0_L2>;	/* appended from boardinfo */
  }; //end cpu@0x0 (hps_0_arm_a9_0)

  hps_0_arm_a9_1: cpu@0x1 {
  	device_type = "cpu";
  	compatible = "arm,cortex-a9-17.0", "arm,cortex-a9";
  	reg = <0x00000001>;
  	next-level-cache = <&hps_0_L2>;	/* appended from boardinfo */
  }; //end cpu@0x1 (hps_0_arm_a9_1)
}; //end cpus

…

sopc0: sopc@0 {
device_type = “soc”;
ranges;
#address-cells = <1>;
#size-cells = <1>;
compatible = “ALTR,avalon”, “simple-bus”;
bus-frequency = <0>;

…
  fpgabridge1: fpgabridge@1 {
  	compatible = "altr,socfpga-lwhps2fpga-bridge";	/* appended from boardinfo */
  	label = "lwhps2fpga";	/* appended from boardinfo */
  	clocks = <&l4_main_clk>;	/* appended from boardinfo */
  	reset-names = "lwhps2fpga";	/* appended from boardinfo */
  	resets = <&hps_0_rstmgr 97>;	/* appended from boardinfo */
  }; //end fpgabridge@1 (fpgabridge1)

  base_fpga_region {
  	compatible = "fpga-region";
  	#address-cells = <0x1>;
  	#size-cells = <0x1>;
  	fpga-mgr = <&hps_0_fpgamgr>;
  	fpga-bridges = <&fpgabridge1>;
  	ranges;
  };
}; //end sopc@0 (sopc0)

chosen {
bootargs = “console=ttyS0,115200”;
}; //end chosen
}; //end /

DTSO:

/dts-v1/;
/plugin/;
/ {
fragment@0 {
target-path = “/sopc@0/base_fpga_region”;
#address-cells = <0x1>;
#size-cells = <0x1>;
overlay {
firmware-name = “ArrowSocKit_AlternativeDesign.rbf”;
};
};
};

Application is described in documentation meant above, i.e
cat /lib/firmware/ArrowSoCKit.dtbo > /configs/device-tree/overlays/AlterantiveDesign/dtbo
while ArrowSocKit_AlternativeDesign.rbf is placed at /lib/firmware.

I hope this will help someone.

mates · December 17, 2020, 10:35pm

@JanKonecny Thank you - your information is quite relevant and helpful. It is still a “learning experience”:
Few sleepless nights passed and now I am getting results: “Kernel Panic” Negative result is also a result, right? I may highly benefit form additional pointers since things might have evolved over the time with new kernels(?)
DE0-NANO-SoC
4.14.130-ltsi-socfpga-r2 (Debian 10 - if that is relevant)

> root@arm:/lib/firmware# dtc -O dtb -o fpga_config.dtbo -b 0 -@ fpga_config.dts
> fpga_config.dtbo: Warning (avoid_unnecessary_addr_size): /fragment@0: unnecessary #address-cells/#size-cells without "ranges" or child "reg" property
mkdir /sys/kernel/config/device-tree/overlays/fpga_config
echo fpga_config.dtbo > /sys/kernel/config/device-tree/overlays/fpga_config/path
[13009.641348] OF: overlay: find target, node: /fragment@0, path '/soc/base_fpga_region' not found
[13009.650092] OF: overlay: init_overlay_changeset() failed, ret = -22
[13009.656344] create_overlay: Failed to create overlay (err=-22)

If anyone is willing to share more recent information and preferably tested overlay file I would appreciate it tremendously. I may even get back to what I wanted to do two weeks ago - start FPGA coding )
I managed to create my own rbf file and make it configure the FPGA portion of the SoC from uboot. My ultimate goal is to read/write fet of FPGA registers from Python. Pointers to any working solutions may allow me to get more sleep than I am getting right now.
(And if Honza doesn’t mind direct messaging - we speak the same language).
Best regards form California and thanks for reading that far.
Martin