Protected materials
Microsoft has given the Azure AI Evaluation SDK one other operate: testing how usually the LLMs underpinning purposes generate responses containing what it calls “protected material” — maybe higher considered forbidden materials, because the class consists of copyright textual content to which the enterprise is unlikely to personal the rights, together with track lyrics, recipes, and articles. To examine for it, the LLM’s outputs are in contrast with an index of third-party textual content content material maintained on GitHub, Thigpen wrote.
“Users can drill into evaluation details to better understand how their application typically responds to these user prompts and the associated risks,” Thingpen defined.
Two APIs are offered: one to flag the output of protected copyright textual content, and one other to flag output of protected code together with software program libraries, supply code, algorithms, and different programming-related supplies.
