We just wrapped up the Oct 5, 2023 AI, Machine Learning and Data Science Meetup, and if you missed it or want to revisit it, here’s a recap! In this blog post you’ll find the playback recordings, highlights from the presentations and Q&A, as well as the upcoming Meetup schedule so that you can join us at a future event.
First, Thanks for Voting for Your Favorite Charity!
In lieu of swag, we gave Meetup attendees the opportunity to help guide a $200 donation to charitable causes. The charity that received the highest number of votes this month was The Coalition for Rainforest Nation (CfRN), an organization working on the responsible stewardship of the world’s last great rainforests to achieve environmental and social sustainability.. We are sending this event’s charitable donation of $200 to CfRN on behalf of the computer vision community!
Automatic Prompt Optimization with “Gradient Descent” and Beam Search
Large Language Models (LLMs) have shown impressive performance but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparametric solution to this problem, Automatic Prompt Optimization (APO), which is inspired by numerical gradient descent to automatically improve prompts, assuming access to training data and an LLM API. Our experiments suggest this method can outperform prior prompt editing techniques and improve an initial prompt’s performance by up to 31%, by using data to rewrite vague task descriptions into more precise annotation instructions.
Reid Pryzant is a Senior Research Scientist at Microsoft, and former Computer Science PhD at Stanford University advised by Dan Jurafsky. His work has won outstanding research awards from CVPR, AAAI, and the National Science Foundation.
Q&A
- Can you define natural language “gradients” in simple terms?
- Is what you are describing a level or two above the vectorized data math where results and errors are measured against a prompt?
- How many iterations or feedback loops are sufficient to get the near perfect prompt?
- Is the same LLM used for gradients and new prompts?
- How good are the prompts compared to a really good human or a human plus a 5 min chat iteratively improving your prompt? What level of quality prompt do you need to start out with?
- How do you define the cost/budget for the API calls before you start running the algorithm?
Resource links
Glacier Monitoring with Computer Vision Models
The temporal variability of marine-terminating glacier front positions provides valuable information on the state of the glaciers. Therefore, the position of these fronts is an important parameter influencing the accuracy of climate models. To obtain the position, satellite imagery has traditionally been analyzed by hand. As the amount of satellite imagery and the need for accurate climate models is increasing, deep learning techniques are applied to extract the glacier front position from satellite images. In this talk, state-of-the-art models for this purpose will be discussed.
Nora Gourmelon is a PhD candidate in Computer Science at the Friedrich-Alexander-Universität Erlangen-Nürnberg working on AI for Earth. Her main focus lies on the segmentation of glacier calving fronts in Synthetic Aperture Radar (SAR) satellite imagery.
Q&A
- How many bands or channels do the radar images have? What about wavelengths?
- Are the radar images available directly and is there any need to process the signals to get the images?
- What were some of the preprocessing techniques that helped in optimization?
- Is glacier melting so quick that the line changes every other day, so the next satellite in orbit has a different image?
- How long did this entire project take? How many people were on the team?
- Are there ways that the model could benefit by including visual images in the cases were you have them?
- Can this model be incorporated into a model that looks at suburban and urban development?
- Isn’t the depth and height of the glacier also an important feature to calculate its melting rate?
- What approach did you utilize to source the benchmark dataset?
Resource links
- Paper: Calving fronts and where to find them: a benchmark dataset and methodology for automatic glacier calving front extraction from synthetic aperture radar imagery
- Browse the CaFFe dataset in your browser
- Blog Post: CaFFe: Calving Fronts and Where to Find Them
- Baseline including the CRF: Conditional Random Fields for Improving Deep Learning-based Glacier Calving Front Delineations
- AMD-HookNet for Glacier Front Segmentation
Build Natural Language Applications with txtai
This talk will introduce txtai and show how it can be used for semantic search, LLM orchestration and language model workflows. An overview of the embeddings database architecture will be discussed along with how vector indexes (sparse and dense), graph networks and relational databases connect together. Example use cases will cover SQL-driven vector search, topic modeling and retrieval augmented generation.
David Mezzetti is the founder of NeuML, the company behind txtai. He is building a suite of open-source, easy-to-use, semantic search and workflow applications. Dave previously co-founded and built Data Works into a 50+ person well-respected software services company leading to a successful acquisition.
Q&A
- What does embeddings really mean? Is there an actual binary of the embedding?
- Can txtai work for api/REST based queries instead of just prompt based manual queries?
Resource links
Join the AI, Machine Learning and Data Science Meetup!
The AI, Machine Learning and Data Science Meetup membership has grown to over 10,000 members! The goal of the Meetups is to bring together communities of data scientists, machine learning engineers, and open source enthusiasts who want to share and expand their knowledge of AI and complementary technologies.
Join one of the 12 Meetup locations closest to your timezone.
- Athens
- Austin
- Bangalore
- Boston
- Chicago
- London
- New York
- Peninsula
- San Francisco
- Seattle
- Silicon Valley
- Toronto
We have exciting speakers already signed up over the next few months! Become a member of the Computer Vision Meetup closest to you, then register for the Zoom.
What’s Next?
Up next on Oct 12 at 10 AM Pacific we have a great line up speakers including:
- Bridging the Gap: Advancing Civil Engineering Inspections with Computer Vision – Johannes Flotzinger, Civil Engineer & Research assistant at Universität der Bundeswehr München
- Adapting to Change: Foundation Models, APIs, and the Past, Present and Future of AI Development – Pietro Bolcato at Kittl
Register for the Zoom here. You can find a complete schedule of upcoming Meetups on the Voxel51 Events page.
Get Involved!
There are a lot of ways to get involved in the Computer Vision Meetups. Reach out if you identify with any of these:
- You’d like to speak at an upcoming Meetup
- You have a physical meeting space in one of the Meetup locations and would like to make it available for a Meetup
- You’d like to co-organize a Meetup
- You’d like to co-sponsor a Meetup
Reach out to Meetup co-organizer Jimmy Guerrero on Meetup.com or ping me over LinkedIn to discuss how to get you plugged in.
The Computer Vision Meetup network is sponsored by Voxel51, the company behind the open source FiftyOne computer vision toolset. FiftyOne enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster. It’s easy to get started, in just a few minutes.
