Welcome to our weekly FiftyOne tips and tricks blog where we cover interesting workflows and features of FiftyOne! This week we are taking a look at plugins! We took some snippets from the recent plugin workshop and brought you the highlights!
Wait, what are FiftyOne Plugins?
FiftyOne Plugins is Voxel51’s newest way to upgrade your computer vision experience. Easily create UI elements to extend the capability of the FiftyOne app with just python code. Create operators to make easy and repeatable workflows that run on your data, all in one place. Download the latest Voxel51 and community plugins to take advantage of features like image quality assurance, zero shot prediction, concept space traversal, and more!
- If you like what you see on GitHub, give the project a star.
- Get started! We’ve made it easy to get up and running in a few minutes.
- Join the FiftyOne Slack community, we’re always happy to help.
Ok, let’s dive into this week’s tips and tricks!
Creating Stunning UIs
Starting off hot, let’s show how you can easily create UI elements with FiftyOne plugins. There’s no better place to start than the Voxel51 Plugin Management and Development plugin! This awesome plugin allows you to easily create customizable elements without ever having to touch JS! To download this plugin or any others, you just need to pass the github repo and the name of the plugin:
fiftyone plugins download \
https://github.com/voxel51/fiftyone-plugins \
--plugin-names @voxel51/plugins
Once it’s installed and you have opened your FiftyOne app, hit the backtick key or the “ ` “ key to open up the operator menu. With the management and development plugin we are able to install, enable, disable, and build plugins! Let’s take a look at how it can help us build a plugin!
Using the build_operator_skeleton plugin, you are able to outline all the expectations for your plugin, from inputs and outputs to name and execution, this plugin will take in all your parameters and create a custom python code just for you! It is an amazing way to get started building any plugin.
In the build_plugin_component operator, you are able to take an even finer grain approach by creating different type of UI elements such as input fields, messages, or dropdowns. The helper plugin will not only show you a python code snippet of the UI element you created, but will also render in a live example below hand!
This plugin greatly accelerates the plugin development process and is a must to anyone new to building FiftyOne plugins!
Add Python Modules to Your Plugin Without Worry
Want to add a python library to your plugin but are worried it may not be installed? Maybe you have a cloned source directory and need to set your path to it in order to run. FiftyOne has you covered on both fronts. You can use fou.lazy_impot(“library”) or add folders directly to your path, all within a plugin.
twilio = fou.lazy_import("twilio")
with add_sys_path(os.path.dirname(os.path.abspath(__file__))):
from twilio_engine import (
get_twilio_sid,
get_twilio_auth_token,
get_twilio_messages,
)
No more headaches or dependency wrangling. FiftyOne makes it simple to add whatever python libraries you are using to your FiftyOne plugin.
Developing JavaScript Plugins
For the full stack or eager developer, you can create custom JavaScript code to add even more UI elements to your plugin. One such example is the panels we use within the app, such as histograms or embeddings. You can create your own custom panel by using JavaScript! To get started, the code in Python is very similar to before. At the execution of our plugin, you can see below we are telling the app to open the “YouTubePlayerPanel”.
class OpenYouTubePanel(foo.Operator): @property def config(self): return foo.OperatorConfig( name="open_youtube_player_panel", label="Open Youtube Player Panel", unlisted=False, ) def execute(self, ctx): ctx.trigger( "open_panel", params=dict( name="YouTubePlayerPanel", isActive=True, layout="horizontal" ), ) def register(p): p.register(OpenYouTubePanel)
We can define this “YouTubePlayerPanel” in a JavaScript file in src/YouTubePlayerPlugin.tsx
registerComponent({
name: "YouTubePlayerPanel",
label: "YouTube Player",
component: YouTubePlayerPanel,
type: PluginComponentType.Panel,
});
The two files will communicate with each other and voila! You have your own custom panel! To see more of this example, check out the youtube plugin here!
Add Node Modules as Dependencies
When working with JavaScript, you are able to bring in custom node modules as well! In this case we can leverage the react-dropzone module to be able to drag and drop images straight into our FiftyOne app. Check it out below!
To accomplish this, just add the module in the “dependencies” section of the package.json file.
"dependencies": {
"@emotion/react": "^11.11.0",
"@emotion/styled": "^11.11.0",
"@mui/icons-material": "^5.11.16",
"@mui/material": "^5.13.0",
"@mui/styled-engine": "^5.12.3",
"@rollup/plugin-node-resolve": "^15.0.2",
"@vitejs/plugin-react": "^4.0.0",
"lodash": "^4.17.21",
"nodemon": "^3.0.1",
"react": "^18.2.0",
"react-dropzone": "^14.2.3",
"react-is": "^18.2.0",
"react-markdown": "^8.0.7",
"styled-components": "^5.3.10",
"vite-plugin-externals": "^0.6.2"
},
Conclusion
I hope this is only the start of your plugin adventure! There are many features we are excited to release soon to level up plugins even more! Be sure to check out the existing plugins over at the FiftyOne Plugins Repo. If you are interested in building your own, be sure to take advantage of the community slack where developers have discussions everyday to build the latest and greatest in computer vision in FiftyOne. Your first FiftyOne plugin is only a PR away!
Join the FiftyOne Community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
- 2,000+ FiftyOne Slack members
- 5,000+ stars on GitHub
- 5,000+ Meetup members
- Used by 370+ repositories
- 60+ contributors
