Turns out when you are connected to an IPv6 network, your subnet will not properly forward IPv4 traffic. The only way around that without having to enable IPv6 on your router and configure all this bullshit is to set up an Exit Node on your Tailscale host.
Then enable the Exit Node in your Tailscale app on your phone or computer and you should be able to reach your subnet devices that are not running Tailscale without any issues now.