Remember Theranos, the company that blew up last year for deceiving investors about the accuracy of their blood testing equipment? Theranos claimed to have developed unique technology that simultaneously miniaturized and improved the speed of blood testing, and investors bought it. When asked about the science, they said they had it and would release it, but never did. Eventually, it became clear that their devices did not actually work as intended (they were inaccurate), and that they had actively deceived the public.
I mention this as an admonition about “public” medical devices—devices developed for and marketed to the public, rather than hospitals and doctors—and the inherent risks they carry. Such devices are often heralded as “game-changers” and marketed as being direct-to-consumer to avoid the stuffy, slow-to-change atmosphere of the medical establishment, but the reality is that it’s this same stuffy atmosphere that protects consumers from devices (and tests, medicines, etc.) that don’t work or aren’t as effective as advertised. If Theranos had attempted to enter the medical sector, it’s likely their ineffective tests would have been detected far sooner or even before it entered the market!
Enter the Nima Sensor
Today, I saw an article about a new device called the “Nima Sensor”, which is a handheld assay that tests foods for gluten. Their pitch is that now people who have celiac disease or are “non-celiac gluten sensitive” (a dubious diagnosis without any formal means of testing and without much evidence to back it) can easily test the foods they eat at home or at restaurants for gluten and have a result in just a few minutes, thereby giving them confidence that their food is truly gluten-free.
Now, I have a few problems with this concept. First, because they’re sidestepping the medical establishment, I have doubts it’ll actually reach the hands of people who could actually benefit from it. Instead of celiacs being able to get the device covered by insurance due to their diagnosis (and otherwise being restricted), it’s open to the public so anyone can purchase it regardless of documented medical need if they have enough money. Consequent to this, it will likely not be seen as a medical device that should be accommodated politely, but rather as another frustration that restaurant staff must deal with from wealthy customers.
In line with this, I also have questions about the sensitivity and specificity of the test, neither of which are mentioned on the website (nor in any peer-reviewed journal). They claim they can detect gluten above 5 parts per million, but with what frequency are these true positives? How often are they false negatives? We have no idea! The only “evidence” they provide is a blog post where they compare results from a Nima Sensor to a more commonly used assay, the Ridascreen(c) Gliadin assay. They claim to be working on third-party validation, but I’ll remain skeptical of such claims until some results materialize; again, if this were an approved medical device, the third-party validation would be a requirement before sales, not an option to be pursued down the line.
Beyond these questions, there’s also the question of the overall reliability of gluten assays in general, which are known to require correction based on the food in question. Basically, these assays test for the presence of either glutenin or gliadin, but these fragments can vary depending on the source of gluten (and even the variety of wheat). The assays also have different affinities for these different particles so they must be corrected to ensure they don’t over-report the presence of gluten (which can skew upwards of 4-10x higher if not corrected). One report comparing the different assays concluded that while they could reliably detect gluten in simple baked goods (e.g., products where the ingredients are few and well-known), “the current data indicate that further work is needed to determine their suitability for processed foods and complex matrixes [sic].” Complex matrices are exactly what will be most frequently tested, and the consumer has no way of correcting (or even knowing how to correct) the results so they’re accurate.
Public Devices Sidestep Peer-Review—But They Shouldn’t
Overall, the trend I see with medical devices and tests marketed to the public is that they are A) expensive; B) unvalidated; and C) tight-lipped on scientific data. They always provide enough information to assuage a casual reader, but never provide the actual data that would allow a scientifically versed individual to assess the quality or accuracy of the device and/or test.
Why is this? We can’t be certain, but I would guess that these companies are really more interested in making a profit than providing a necessary good or service. If Nima was confident in their product, they would take the steps to classify their product as a medical device, get it FDA-approved, and allow diagnosed celiacs to get it covered through their insurance. Instead, they have a bunch of lines in their help database about how they might pursue such classification in the future, but haven’t yet. Probably they realized there’s more money and less regulatory tape in the public sphere, and so they took a shortcut. Could Nima convince me otherwise by releasing better data, getting third-party validation, and going through the steps to qualify their product for insurance? Sure. But I’m not going to hold my breath.
In the end, this isn’t a post just about Nima; it’s about the countless pseudomedical and pseudoscientific products out there. If a device or test claims to do something that normally you would have to go to the doctor to do—and says it’ll do it better, faster, or more cheaply—you should be skeptical. Really, you should be skeptical in general because these products don’t have a good track record, no matter how often they claim to revolutionize the industry.
