[Fizzlemizz]

You can't unkook something, just alter what the hook function does,

You could use OnUpdate on a frame of your own and when the CUSTOMFRAME is found, hide your frame (that stops it's OnUpdate from running) or set it's OnUpdate script to nil.

Maybe something like:

local T = C_Timer.NewTicker(1, function()
    if CUSTOMFRAME then
        CUSTOMFRAME:HookScript("OnUpdate", function(self)
            print("have it")
        end)
        T:Cancel()
    end
end)

There are probably other possabilities depending on what other conditions might exist

Is the frame in a Load-On-Demand addon?
Is there another frame/function that creates the frame that you can hook?
...???