Resolving 'Unable handle kernel paging request at virtual address' on a custom NiosII Cyclone III board

Hello All,

I am trying to move from 2.6.28 kernel to 3.10. I have used the NiosII tutorial that uses socfpga/buildroot. I am pretty confident that there must me something I am doing wrong but I just do not see it. I can get all the way to a prompt but after few seconds/minutes (seems random) I get the following error

Unable handle kernel paging request at virtual address 0d400000. I do not have anything assigned on that address in my sopc build.

Below is a complete boot log.

Linux version 3.10.87-ltsi-rt93 (yg@ubuntu) (gcc version 4.7.3 (Sourcery CodeBench Lite 2013.05-43) ) #189 PREEMPT Wed Dec 26 15:09:49 EST 2018
early_console initialized at 0xe5000060
Nope
Warning: dcache size configuration mismatch (0x800 vs 0x1000) of CONFIG_NIOS2_DCACHE_SIZE vs device tree dcache-size
On node 0 totalpages: 57344
free_area_init_node: node 0, pgdat cc3e2224, node_mem_map cc407a60
  Normal zone: 448 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 57344 pages, LIFO batch:15
pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 56896
Kernel command line: debug console=ttyJ0,115200
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Sorting __ex_table...
Memory available: 26608k/4114k RAM (1860k kernel code, 2253k data)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
NR_IRQS:64 nr_irqs:64 0
Calibrating delay loop... 78.64 BogoMIPS (lpj=393216)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
NET: Registered protocol family 16
Switching to clocksource timer
FS-Cache: Loaded
NET: Registered protocol family 2
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 3, 40960 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 12288 bytes)
UDP-Lite hash table entries: 256 (order: 1, 12288 bytes)
msgmni has been set to 51
ttyJ0 at MMIO 0x5000060 (irq = 30) is a Altera JTAG UART
TCP: cubic registered
turn off boot console early0
Freeing unused kernel memory: 1696K (cc1d2000 - cc37a000)
# Unable handle kernel paging request at virtual address 0d400000
Unable handle kernel paging request at virtual address 0d400000
Unable handle kernel paging request at virtual address 0d400000
Unable handle kernel paging request at virtual address 0d400000

My DTS looks like this

/*
 * This devicetree is generated by sopc2dts version rel_14.0_RC3-38-g2be4c1b on Wed Dec 26 14:47:17 EST 2018
 * Sopc2dts is written by Walter Goossens <waltergoossens@home.nl>
 * in cooperation with the nios2 community <nios2-dev@lists.rocketboards.org>
 */
/dts-v1/;

/ {
	model = "ALTR,sopc_top";
	compatible = "ALTR,sopc_top";
	#address-cells = <1>;
	#size-cells = <1>;

	cpus {
		#address-cells = <1>;
		#size-cells = <0>;

		apu: cpu@0 {
			device_type = "cpu";
			compatible = "ALTR,nios2-12.1", "ALTR,nios2-1.0";
			reg = <0x00000000>;
			interrupt-controller;
			#interrupt-cells = <1>;
			clock-frequency = <160000000>;	/* embeddedsw.CMacro.CPU_FREQ type NUMBER */
			dcache-line-size = <32>;	/* embeddedsw.CMacro.DCACHE_LINE_SIZE type NUMBER */
			icache-line-size = <32>;	/* embeddedsw.CMacro.ICACHE_LINE_SIZE type NUMBER */
			dcache-size = <4096>;	/* embeddedsw.CMacro.DCACHE_SIZE type NUMBER */
			icache-size = <4096>;	/* embeddedsw.CMacro.ICACHE_SIZE type NUMBER */
			ALTR,implementation = "fast";	/* embeddedsw.CMacro.CPU_IMPLEMENTATION type STRING */
			ALTR,pid-num-bits = <10>;	/* embeddedsw.CMacro.PROCESS_ID_NUM_BITS type NUMBER */
			ALTR,tlb-num-ways = <16>;	/* embeddedsw.CMacro.TLB_NUM_WAYS type NUMBER */
			ALTR,tlb-num-entries = <256>;	/* embeddedsw.CMacro.TLB_NUM_ENTRIES type NUMBER */
			ALTR,tlb-ptr-sz = <8>;	/* embeddedsw.CMacro.TLB_PTR_SZ type NUMBER */
			ALTR,has-div;	/* embeddedsw.CMacro.HARDWARE_DIVIDE_PRESENT type NUMBER */
			ALTR,has-mul;	/* embeddedsw.CMacro.HARDWARE_MULTIPLY_PRESENT type NUMBER */
			ALTR,has-initda;	/* embeddedsw.CMacro.INITDA_SUPPORTED type BOOLEAN */
			ALTR,reset-addr = <0xc0000800>;	/* embeddedsw.CMacro.RESET_ADDR type NUMBER */
			ALTR,fast-tlb-miss-addr = <0xce100000>;	/* embeddedsw.CMacro.FAST_TLB_MISS_EXCEPTION_ADDR type NUMBER */
			ALTR,exception-addr = <0xcc000020>;	/* embeddedsw.CMacro.EXCEPTION_ADDR type NUMBER */
		}; //end cpu@0 (apu)
	}; //end cpus

	memory {
		device_type = "memory";
		reg =  <0x0c000000 0x02000000>,
			   <0x0e100000 0x00001000>;
	}; //end memory

	sopc0: sopc@0 {
		device_type = "soc";
		ranges;
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "ALTR,avalon", "simple-bus";
		bus-frequency = <160000000>;

		clock_crossing_prot: bridge@c000000 {
			compatible = "ALTR,avalon-12.1", "simple-bus";
			reg = <0x0c000000 0x02000000>;
			#address-cells = <1>;
			#size-cells = <1>;
			ranges = <0x00000000 0x0c000000 0x02000000>;

		}; //end bridge@c000000 (clock_crossing_prot)

		epcs_controller: spi@c00 {
			compatible = "ALTR,spi-12.1", "ALTR,spi-1.0";
			reg = <0x00000c00 0x00000400>;
			interrupt-parent = <&apu>;
			interrupts = <5>;
			bank-width = <2>;
			device-width = <1>;
			#address-cells = <1>;
			#size-cells = <0>;

			m25p80@0 {
				compatible = "m25p80";
				spi-max-frequency = <25000000>;
				reg = <0>;
			}; //end m25p80@0
		}; //end spi@c00 (epcs_controller)

		high_res_timer: timer@400b000 {
			compatible = "ALTR,timer-12.1", "ALTR,timer-1.0";
			reg = <0x0400b000 0x00000020>;
			interrupt-parent = <&apu>;
			interrupts = <3>;
			clock-frequency = <160000000>;
		}; //end timer@400b000 (high_res_timer)

		sysid: sysid@400b020 {
			compatible = "ALTR,sysid-12.1", "ALTR,sysid-1.0";
			reg = <0x0400b020 0x00000008>;
		}; //end sysid@400b020 (sysid)

		igor_mac: ethernet@4009000 {
			compatible = "opencores,ethoc-1.0", "opencores,ethoc";
			reg = <0x04009000 0x00001000>;
			interrupt-parent = <&apu>;
			interrupts = <2>;
			address-bits = <48>;
			max-frame-size = <1518>;
			local-mac-address = [00 00 00 00 00 00];
		}; //end ethernet@4009000 (igor_mac)

		igor_mac_2: ethernet@1000 {
			compatible = "opencores,ethoc-1.0", "opencores,ethoc";
			reg = <0x00001000 0x00001000>;
			interrupt-parent = <&apu>;
			interrupts = <4>;
			address-bits = <48>;
			max-frame-size = <1518>;
			local-mac-address = [00 00 00 00 00 00];
		}; //end ethernet@1000 (igor_mac_2)

		bridge_to_slow_peripherals: bridge@6000000 {
			compatible = "ALTR,avalon-12.1", "simple-bus";
			reg = <0x06000000 0x00000800>;
			#address-cells = <1>;
			#size-cells = <1>;
			ranges = <0x00000100 0x06000100 0x00000010>,
				<0x00000110 0x06000110 0x00000010>,
				<0x00000130 0x06000130 0x00000010>,
				<0x00000140 0x06000140 0x00000010>,
				<0x00000120 0x06000120 0x00000010>,
				<0x000000e0 0x060000e0 0x00000020>,
				<0x000000c0 0x060000c0 0x00000020>,
				<0x000000a0 0x060000a0 0x00000020>,
				<0x00000160 0x06000160 0x00000010>,
				<0x00000170 0x06000170 0x00000010>,
				<0x000001a0 0x060001a0 0x00000010>,
				<0x000001c0 0x060001c0 0x00000020>,
				<0x00000220 0x06000220 0x00000020>,
				<0x00000240 0x06000240 0x00000020>,
				<0x00000260 0x06000260 0x00000020>,
				<0x00000280 0x06000280 0x00000020>,
				<0x000002b0 0x060002b0 0x00000010>,
				<0x00000710 0x06000710 0x00000010>,
				<0x00000700 0x06000700 0x00000010>,
				<0x000002a0 0x060002a0 0x00000010>;

		}; //end bridge@6000000 (bridge_to_slow_peripherals)

		jtag_uart_0: serial@5000060 {
			compatible = "ALTR,juart-12.1", "ALTR,juart-1.0";
			reg = <0x05000060 0x00000008>;
			interrupt-parent = <&apu>;
			interrupts = <30>;
		}; //end serial@5000060 (jtag_uart_0)

	}; //end sopc@0 (sopc0)

	chosen {
		bootargs = "debug console=ttyJ0,115200";
	}; //end chosen
}; //end /

Below are my memory setting.

What am I missing? Where can I go from here?

So I got 4.14 running on the board which has more info on the panic

Unable to handle kernel paging request at virtual address 0d400000
ea = c000089c, ra = cc037448, cause = 15

Its the same memory space as with the 3.10 kernel. I do not use anything in the 0d400000 address which is confusing. System Map is linked

[https://drive.google.com/open?id=1p5_-8HwO1eJ6MYN-4jE6FhZB6PeFktfv](http://System Map link )

Complete boot

Using cable "USB-Blaster [USB-1]", device 1, instance 0x00
Pausing target processor: OK
Initializing CPU cache (if present)
OK
Downloaded 3411KB in 19.6s (174.0KB/s)
Verified OK
Starting processor at address 0xCC500000
nios2-terminal: connected to hardware target using JTAG UART on cable
nios2-terminal: "USB-Blaster [USB-1]", device 1, instance 0
nios2-terminal: (Use the IDE stop button or Ctrl-C to terminate)

Linux version 4.14.59 (ubuntu) (gcc version 7.3.1 20180222 (Sourcery CodeBench Lite 2018.05-5)) #42 PREEMPT Thu Dec 27 09:40:13 EST 2018
On node 0 totalpages: 57344
free_area_init_node: node 0, pgdat cc4b4acc, node_mem_map cc4ed960
  Normal zone: 448 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 57344 pages, LIFO batch:15
pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
pcpu-alloc: [0] 0
Built 1 zonelists, mobility grouping on.  Total pages: 56896
Kernel command line: debug console=ttyJ0,115200 vmalloc=2M
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Sorting __ex_table...
Memory: 25696K/229376K available (2065K kernel code, 80K rwdata, 392K rodata, 2280K init, 194K bss, 203680K reserved, 0K cma-reserved)
Preemptible hierarchical RCU implementation.
        Tasks RCU enabled.
NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
clocksource: nios2-clksrc: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 11945377789 ns
Console: colour dummy device 80x25
Calibrating delay loop (skipped), value calculated using timer frequency.. 320.00 BogoMIPS (lpj=1600000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
Hierarchical SRCU implementation.
cpu cpu0: Error -2 creating of_node link
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: -1, 3072 bytes)
clocksource: Switched to clocksource nios2-clksrc
workingset: timestamp_bits=30 max_order=13 bucket_order=0
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
io scheduler mq-deadline registered
io scheduler kyber registered
5000060.serial: ttyJ0 at MMIO 0x5000060 (irq = 30, base_baud = 0) is a Altera JTAG UART
console [ttyJ0] enabled
random: get_random_bytes called from init_oops_id+0x54/0x64 with crng_init=0
dmesg

Freeing unused kernel memory: 2280K
This architecture does not have kernel memory protection.
mount (17) used greatest stack depth: 7236 bytes left
mount: mounting tmpfs on /dev/shm failed: Invalid argumentmount (20) used greatest stack depth: 7208 bytes left

mount: mounting tmpfs on /tmp failed: Invalid argument
mount: mounting tmpfs on /run failed: Invalid argument
random: fast init done
Starting logging: start-stop-daem (29) used greatest stack depth: 7184 bytes left
OK
S01logging (28) used greatest stack depth: 6896 bytes left
Initializing random number generator... random: dd: uninitialized urandom read (512 bytes read)
done.
Starting network: ip: socket: Function not implemented
ip: socket: Function not implemented
FAIL

Welcome to Nios II
nios2 login: Unable to handle kernel paging request at virtual address 0d400000
ea = c000089c, ra = cc037474, cause = 15
Kernel panic - not syncing: Oops

Issue Resolved a watch dog was corrupting memory in that space.