I have a friend who wants to use exactly that technology for an immersive multi-source data fusion environment. It has promise but there are also some open source alternatives that look interesting like Open Croquet from Alan Kay and company.