net: xilinx: Remove .owner field for driver