Yep. We have that situation now, in fact. There are IA64 versions of Windows 2000 and Linux, and software must be specially compiled for that chip, exactly like software for an entirely different architecture (e.g. Alpha, PPC, UltraSparc).
x86-64 will run current software, but current chips...