New Delhi: There is no doubt that AI has begun helping researchers and scientists across the world. But a recent study argues that academia is delegating responsibility and trust to tools that researchers cannot understand, creating “scientific black boxes”.
The study, “The black-box future of ecology and conservation” published in BioScience on 15 August was conducted by a team of international researchers. They said that although AI might be useful for understanding biodiversity loss by looking through large datasets, if conservationists no longer understand how their tools come to conclusions, then conservation related decisions could be made on the basis of opaque machines.
“Many of these tools represent true black boxes, by keeping the processes behind those results largely hidden. They are often owned by private companies that intentionally limit access to information about how their systems operate or process data, guided by proprietary constraints and commercial aims,” said Ivan Jarić, professor of ecology at the University of Paris-Saclay, and lead author of the study, in a press release.
By “black box” the study’s authors refer to a system which cannot be independently inspected, evaluated, reproduced, or even understood by the people relying on it. This is particularly true in the field of ecology where researchers now work with large amounts of data — satellite imagery, camera traps, acoustic sensors, DNA, climate models. This gives researchers a lot of information to work with but it makes them dependent on computational systems.
In such cases, AI is much better at identifying patterns which individual researchers could miss. AI systems could identify species in camera trap images, recognise animal calls, detect changes in vegetation based on satellite imagery, and predict where endangered species could be found. Much of this information could be used to make critical decisions regarding habitat restoration, conservation funding. However, the study questions whether these would be sound decisions if they are based on conclusions that cannot be verified.
This becomes particularly consequential since AI cannot even out potential data bias in ecological data. If monitoring data comes from easily accessible areas, wealthy countries, popular species, regions with a lot of researchers, or even places with good internet, it could create skewed results.
“Human oversight should remain central throughout the research process, especially since it is the study authors who must take responsibility for any errors and uncertainties produced by the use of black-box tools in their work,” said co-author Michael Bertram, associate professor at the Swedish University of Agricultural Sciences.