Boston AI, ML, and Computer Vision Meetup

The widespread deployment of general-purpose AI (GPAI) systems introduces significant new risks. Yet the infrastructure, practices, and norms for reporting flaws in GPAI systems remain seriously underdeveloped, lagging far behind more established fields like software security. Based on a collaboration between experts from the fields of software security, machine learning, law, social science, and policy, we identify key gaps in the evaluation and reporting of flaws in GPAI systems. We call for three interventions to advance system safety.

First, we propose using standardized AI flaw reports and rules of engagement for researchers in order to ease the process of submitting, reproducing, and triaging flaws in GPAI systems. Second, we propose GPAI system providers adopt broadly-scoped flaw disclosure programs, borrowing from bug bounties, with legal safe harbors to protect researchers. Third, we advocate for the development of improved infrastructure to coordinate distribution of flaw reports across the many stakeholders who may be impacted. By promoting robust reporting and coordination in the AI ecosystem, these proposals could significantly improve the safety, security, and accountability of GPAI systems.

The old adage “it takes a village” took on a new meaning when Rizel became a parent. She realized that raising a child to their full potential requires more than what one person alone can provide. It takes a connected community of support. Similarly, AI tools, while valuable, have a bigger impact when they’re interconnected with systems in our daily workflows.

Model Context Protocol (MCP) is an open standard that removes silos between AI applications and other apps, enabling them to share context and capabilities instead of working in isolation.

Join Rizel as she demonstrates how Goose, an open source local AI agent, uses multiple MCP servers to handle complex tasks across your workflow. Audience members will learn how to build their own digital village where AI and everyday applications augment each other’s capabilities – because just like raising a child, the best results come from a connected community

In the domain of RAG, integrating diverse data modalities, such as text, images, and tables, presents both challenges and opportunities. This talk delves into the application of ColPali, a cutting-edge Vision Language Model, to enhance multimodal retrieval within RAG systems. By leveraging ColPali’s ability to generate multi-vector embeddings directly from document images, we can bypass traditional OCR and complex preprocessing pipelines, leading to more efficient and accurate retrieval of visually rich documents.

We will explore the architecture of ColPali, its integration into RAG workflows, and demonstrate its efficacy in handling documents that combine textual and visual information. Attendees will gain insights into implementing ColPali for improved document retrieval and the broader implications for multimodal AI applications.