Ethernet issue. No PHY found

Hi,

I am using a custom board based on the Cyclone V socfpga and socfpga-5.0 linux kernel. I am also routing the HPS EMAC through the FPGA as described in this link Cyclone V RGMII Example Design | Projects | RocketBoards.org

I am unable to get the ethernet up and running.

The relevant dmesg that I get concerning the ethernet is;

modprobe dwmac-generic.ko

[ 360.815734] stmmaceth ff700000.ethernet: snps,phy-addr property is deprecated
[ 360.826301] stmmaceth ff700000.ethernet: PTP uses main clock
[ 360.832557] stmmaceth ff700000.ethernet: User ID: 0x10, Synopsys ID: 0x37
[ 360.839374] stmmaceth ff700000.ethernet: DWMAC1000
[ 360.844262] stmmaceth ff700000.ethernet: DMA HW capability register supported
[ 360.851388] stmmaceth ff700000.ethernet: RX Checksum Offload Engine supported
[ 360.858515] stmmaceth ff700000.ethernet: COE Type 2
[ 360.863387] stmmaceth ff700000.ethernet: TX Checksum insertion supported
[ 360.870080] stmmaceth ff700000.ethernet: Enhanced/Alternate descriptors
[ 360.876683] stmmaceth ff700000.ethernet: Enabled extended descriptors
[ 360.883117] stmmaceth ff700000.ethernet: Ring mode enabled
[ 360.888595] stmmaceth ff700000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[ 360.903795] libphy: stmmac: probed
[ 360.907222] stmmaceth ff700000.ethernet: No PHY found
[ 360.912598] stmmaceth ff700000.ethernet: stmmac_dvr_probe: MDIO bus (id: 0) registration failed

modprobe dwmac-altr-socfpga.ko

[ 1818.241624] socfpga-dwmac ff700000.ethernet: snps,phy-addr property is deprecated
[ 1818.249345] socfpga-dwmac ff700000.ethernet: PTP uses main clock
[ 1818.255460] socfpga-dwmac ff700000.ethernet: No sysmgr-syscon node found
[ 1818.262162] socfpga-dwmac ff700000.ethernet: Unable to parse OF data
[ 1818.268575] socfpga-dwmac: probe of ff700000.ethernet failed with error -524

I also didn’t manage to get the ethernet running with the kernel 4.14.

Here is my device tree https://0bin.net/paste/hChzCJ91lCL0GoS9#a9EqMvfcLlYyXpyC1tkHT-sIcGMeikWmI4UCfUryimx

I would be very grateful if someone can enlighten me on what is happening.

Kind Regards,

Deva

Hello @deva,
I suppose the linked article is written for quite old Linux kernel verison. The necessary content of Device Tree is changing, so Linux 5.0 needs a bit different form.
I recommend not to use node of ‘mdio0’ more over with empty node 'ethernet-phy@1’. Current version of the driver for ethernet controller at Intel Cyclone V is able to find its mdio as well as PHY properly without specifying them at Device Tree.
Adding “Micrel,KSZ9021” into the compatibility string ot ethernet controller is wrong. This compatibility string would be at PHY node only (i.e. Yours ‘*ethernet-phy@1’).
Try to use version of ‘ethernet@ff700000’ node which is produced by ‘sopc2dtc’ tool of Intel SoC EDS and, if needed, add necessary properties according to Device Tree bindings https://github.com/altera-opensource/linux-socfpga/blob/master/Documentation/devicetree/bindings/net/socfpga-dwmac.txt.
You could mention that there is no mdio nor PHY nodes, just ‘phy-mode’ property.
Yours Jan Konečný.

Hi Jan

Thank you for your reply! I have made the changes as you have said and this is now my new device tree. https://0bin.net/paste/gDXroNnuSHhorY2s#CZ5lqGu3MAV11CiLNZ85ralTkbeqjxY+SqPn82okHIA

Now it seems the PHY is detected and the correct driver is loaded but I don’t think it is fully solved yet.

# modprobe mii.ko
# lsmod
Module                  Size  Used by    Tainted: G
mii                    16384  0
fpga_alg_a_kernel      16384  0
fpgadatacapture_kernel    16384  0
mcu_kernel             16384  0
uio_pdrv_genirq        16384  0
# cd ethernet/stmicro/stmmac/
# ls
dwmac-altr-socfpga.ko  stmmac-platform.ko
dwmac-generic.ko       stmmac.ko
# modprobe dwmac-generic.ko
[ 2637.317328] stmmaceth ff700000.ethernet: snps,phy-addr property is deprecated
[ 2637.324568] stmmaceth ff700000.ethernet: PTP uses main clock
[ 2637.330817] stmmaceth ff700000.ethernet: Version ID not available
[ 2637.336927] stmmaceth ff700000.ethernet:     DWMAC1000
[ 2637.341838] stmmaceth ff700000.ethernet: DMA HW capability register supported
[ 2637.348962] stmmaceth ff700000.ethernet: RX Checksum Offload Engine supported
[ 2637.356089] stmmaceth ff700000.ethernet: COE Type 2
[ 2637.360960] stmmaceth ff700000.ethernet: TX Checksum insertion supported
[ 2637.367652] stmmaceth ff700000.ethernet: Enhanced/Alternate descriptors
[ 2637.374254] stmmaceth ff700000.ethernet: Extended descriptors not supported
[ 2637.381196] stmmaceth ff700000.ethernet: Ring mode enabled
[ 2637.399910] libphy: stmmac: probed
[ 2637.403341] Micrel KSZ9021 Gigabit PHY stmmac-0:01: attached PHY driver [Micrel KSZ9021 Gigabit PHY] (mii_bus:phy_addr=stmmac-0:01, irq=POLL)
# [ 2637.712906] Micrel KSZ9021 Gigabit PHY stmmac-0:01: attached PHY driver [Micrel KSZ9021 Gigabit PHY] (mii_bus:phy_addr=stmmac-0:01, irq=POLL)
[ 2637.728371] stmmaceth ff700000.ethernet eth0: No Safety Features support found
[ 2637.735887] stmmaceth ff700000.ethernet eth0: registered PTP clock
[ 2639.778387] stmmaceth ff700000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
[ 2639.786674] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
ifconfig
eth0      Link encap:Ethernet  HWaddr 70:B3:D5:5F:DF:FD
           inet6 addr: fe80::72b3:d5ff:fe5f:dffd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:1892 (1.8 KiB)
          Interrupt:29 Base address:0x8000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

# modprobe dwmac-altr-socfpga.ko
# lsmod
Module                  Size  Used by    Tainted: G
dwmac_altr_socfpga     16384  0
dwmac_generic          16384  0
stmmac_platform        16384  2 dwmac_altr_socfpga,dwmac_generic
stmmac                114688  2 dwmac_generic,stmmac_platform
mii                    16384  0
fpga_alg_a_kernel      16384  0
fpgadatacapture_kernel    16384  0
mcu_kernel             16384  0
uio_pdrv_genirq        16384  0
# ifup -a
udhcpc: started, v1.31.0
udhcpc: sending discover
udhcpc: sending discover
udhcpc: sending discover
leasefail.script: Leasefail called
udhcpc: no lease, failing
Failed to bring up eth0.

Even if I try to set a static ip it will not be able to ping any other devices or website or vice versa. It seems that the connection/path is broken somewhere.

Anything I could be missing or investigating?

Kind Regards,
Devashish

Hello @deva,
so, I use very similar hardware and Your Device Tree differs from mine at these parameters for Micrel KSZ-9021 PHY only:

txc-skew-ps = <3000>;
rxc-skew-ps = <3000>;
txen-skew-ps = <0>;
rxdv-skew-ps = <0>;
rxd0-skew-ps = <0>;
rxd1-skew-ps = <0>;
rxd2-skew-ps = <0>;
rxd3-skew-ps = <0>;
txd0-skew-ps = <0>;
txd1-skew-ps = <0>;
txd2-skew-ps = <0>;
txd3-skew-ps = <0>;

These parameters come from a board-specific file ‘board.xml’ for Arrow SoCkit board for sopc2dts tool.
Moreover, are You sure that phy-mode is really MII? Qsys for Cyclone V (at least SX which I use) let me set RGMII only for FPGA implementation of Emac0.
Best wishes.
Your Jan Konečný.

Hi Jan,

So far I have tried literally hundreds of device trees with the skew values, different phy-mode with mii, rgmii and rgmii-id, and even used the sopc2dts tool to generate a device tree(albeit with the board info for the devkit because we don’t have any other board info file and we have followed closely the design of the dev kit) but the issue is still not solved.

I am thinking the problem might be with the “no sysmgr-syscon node found”.

Could it be because we are generating an FPGA file with the an old quartus (14.1)?

[    3.778286] socfpga-dwmac ff700000.ethernet: snps,phy-addr property is deprecated
[    3.785893] socfpga-dwmac ff700000.ethernet: PTP uses main clock
[    3.791938] socfpga-dwmac ff700000.ethernet: No sysmgr-syscon node found
[    3.798639] socfpga-dwmac ff700000.ethernet: Unable to parse OF data
[    3.805030] socfpga-dwmac: probe of ff700000.ethernet failed with error -524
[    3.812285] stmmaceth ff700000.ethernet: snps,phy-addr property is deprecated
[    3.819484] stmmaceth ff700000.ethernet: PTP uses main clock
[    3.825341] stmmaceth ff700000.ethernet: Version ID not available
[    3.831460] stmmaceth ff700000.ethernet:     DWMAC1000
[    3.836336] stmmaceth ff700000.ethernet: DMA HW capability register supported
[    3.843467] stmmaceth ff700000.ethernet: RX Checksum Offload Engine supported
[    3.850586] stmmaceth ff700000.ethernet: COE Type 2
[    3.855456] stmmaceth ff700000.ethernet: TX Checksum insertion supported
[    3.862140] stmmaceth ff700000.ethernet: Enhanced/Alternate descriptors
[    3.868738] stmmaceth ff700000.ethernet: Extended descriptors not supported
[    3.875681] stmmaceth ff700000.ethernet: Ring mode enabled
[    3.891000] libphy: stmmac: probed
[    3.894411] Micrel KSZ9021 Gigabit PHY stmmac-0:01: attached PHY driver [Micrel KSZ9021 Gigabit PHY] (mii_bus:phy_addr=stmmac-0:01, irq=POLL)

Hello @deva,
yes, using Quartus II 14.1 and Linux kernel 5.0 could be source of problem. Quartus (especialy Qsys) generates basic parameters for sopc2dts tool. This parameters are placed at *.sopcinfo file which is input for sopc2dtc and You could see a comment similar to the following one for these parameters at produced dts file:

/* embeddedsw.dts.params.xxx */

However, I have to use the following item at my board.xml file to add the parameter altr,sysmgr-syscon:

<DTAppend name="altr,sysmgr-syscon" parentlabel="HPS_gmac1">
<val type="phandle">HPS_sysmgr</val>
<val type="hex">0x60</val>
<val type="number">2</val>
</DTAppend>

As You can see, I use gmac1, HPS_sysmgr is name of Sytem Manager node and numbers could be different for gmac0 too (I am not sure). Unfortunatelly, I could not remmember where I found it.
Best wishes
Yours JanKonečný.

Hi Jan,

Are you using the emac splitter IP as well for your ethernet connection?

Is it possible to share your dts please?

We have actually tried using Quartus 17 to generate the board info files and the sopc2dts tool from SoC EDS 17 but we get these errors;

$ sopc2dts --input soc_system_new_info.sopcinfo --output socfpga.dtb --type dtb --board board_new.xml --clocks [Fatal Error] :2:2: The markup in the document following the root element must be well-formed. Exception occurred: use -v for more information Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component pll_0 of class altera_pll is unknown Component fpga_0 of class fpga is unknown Component gmii_to_rgmii_adapter_0 of class altera_gmii_to_rgmii_adapter is unknown Component pll_0 of class altera_pll is unknown

Thank you for your help so far!

Kind Regards,
Deva

Hello @deva,
no, I do not use emac splitter.
I tried to upload my DTS file, but it is not possible with forum API and it is too long for simple pasting. Part with node for emac1 of my DTS file follows:

HPS_gmac1: ethernet@0xff702000 {
    compatible = "synopsys,dwmac-17.0", "altr,socfpga-stmmac", "snps,dwmac-3.70a", "snps,dwmac";
    reg = <0xff702000 0x00002000>;
    interrupt-parent = <&HPS_arm_gic_0>;
    interrupts = <0 120 4>;
    clocks = <&emac1_clk>;
    clock-names = "stmmaceth";	/* embeddedsw.dts.params.clock-names type STRING */
    interrupt-names = "macirq";	/* embeddedsw.dts.params.interrupt-names type STRING */
    rx-fifo-depth = <4096>;	/* embeddedsw.dts.params.rx-fifo-depth type NUMBER */
    snps,multicast-filter-bins = <256>;	/* embeddedsw.dts.params.snps,multicast-filter-bins type NUMBER */
    snps,perfect-filter-entries = <128>;	/* embeddedsw.dts.params.snps,perfect-filter-entries type NUMBER */
    status = "okay";	/* embeddedsw.dts.params.status type STRING */
    tx-fifo-depth = <4096>;	/* embeddedsw.dts.params.tx-fifo-depth type NUMBER */
    address-bits = <48>;
    max-frame-size = <3800>;	/* appended from boardinfo */
    local-mac-address = [00 00 00 00 00 00];
    phy-mode = "rgmii";	/* appended from boardinfo */
    phy-addr = <0xffffffff>;	/* appended from boardinfo */
    txc-skew-ps = <3000>;	/* appended from boardinfo */
    rxc-skew-ps = <3000>;	/* appended from boardinfo */
    txen-skew-ps = <0>;	/* appended from boardinfo */
    rxdv-skew-ps = <0>;	/* appended from boardinfo */
    rxd0-skew-ps = <0>;	/* appended from boardinfo */
    rxd1-skew-ps = <0>;	/* appended from boardinfo */
    rxd2-skew-ps = <0>;	/* appended from boardinfo */
    rxd3-skew-ps = <0>;	/* appended from boardinfo */
    txd0-skew-ps = <0>;	/* appended from boardinfo */
    txd1-skew-ps = <0>;	/* appended from boardinfo */
    txd2-skew-ps = <0>;	/* appended from boardinfo */
    txd3-skew-ps = <0>;	/* appended from boardinfo */
    altr,sysmgr-syscon = <&HPS_sysmgr 0x00000060 2>;	/* appended from boardinfo */
    reset-names = "stmmaceth";	/* appended from boardinfo */
    resets = <&HPS_rstmgr 33>;	/* appended from boardinfo */
}; //end ethernet@0xff702000 (HPS_gmac1)

I suppose that errors are caused by node fpga_0 refered at Your board_new.xml file as a parent for node altera_gmii_to_rgmii_adapter_0. Is there something like that? FPGA is probabbly named differently in the soc_system_new_info.sopcinfo file. Or there might be an inconsistence in Quartus Prime 17.

A little note: sopc2dts command should be called with one more parameter
--bridge-removal all

Hi Jan,

Thank you for pasting the relevant part of your DTS, it definitely helped us. The furthest point we have reached so far is getting the driver to load and getting the internet connection through but it only works in one case as described below. Also some other observations are written below;

Note we have 2 sectors on the flash where we can store images.

  1. If we boot from the 1st sector (containing kernel version 3.10-ltsi), ethernet connection and access to the internet is established successfully. If we do a warm reboot (i.e rebooting just the SoC) and boot to sector 2 (containing the linux kernel version 4.14.73-ltsi) the ethernet connection and access to the internet is established successfully.

  2. if we keep warm rebooting the unit the internet connection is always established properly.

  3. If we do a cold reset (i.e resetting the EMAC, QSPI etc… just like a pulling the power plug) and boot straight to the linux kernel 4.14.73-ltsi then the ethernet connection is established but when dhcp runs we get the error message “Failed to bring up eth0”. Setting a static IP does not work either. So no packets get out!

  4. Putting down eth0 by using commands like (ifconfig eth0 down or ifdown --force eth0) will cause the system to immediately COLD RESET! which is very strange.

So it seems that when 3.10 configures communication to the EMAC, which 4.14 can just use for internet communication but when it tries to setup the EMAC communication from scratch it fails and somehow is now linked to a reset pin on the SoC.

This is quite a head scratching situation.

Any ideas?

Kind Regards,

Deva

Hello @deva,
I am very sorry, I have no ideas.
Try to confirm all register addresses with Intel Cyclone V Hard Processor System Technical Reference Manual.
Best wishes and good luck.
Jan Konečný.

Hi Jan,

I have managed to make it work by changing the values of the rxc and txc skew values in the device tree to:
txc-skew-ps = <0x00000a28>;
rxc-skew-ps = <0x000007d0>;
It is actually some values used by somebody who I happened to find but I am wondering how I should have proceeded to find these values myself. Do you have an idea?

Hello @deva,
congratulation for solving Your problem! :+1: This seems to be very difficult to find it.
I really do not know what values of skew should be used. Of course, they have to be specified at datasheet of Your PHY at chapter of timing
https://www.microchip.com/wwwproducts/en/KSZ9021#datasheet-toggle.
You have mentioned using splitter

The splitter might change timing requirements for PHY. I am sorry, I do not understand these themes.

Anyway, good job and good luck in advance.
Yours Jan Konečný.