Towards the end of last year, Meta released SAM 3, which stands for the third version of its Segment Anything Model. The way it generally works is that it allows you to detect, edit, and experiment with things in images and videos. For example, if you were looking at a video of a street, you could ask it to find all the scooters (which I did below), count the number of pedestrians wearing black pants, blur all the license plates on the cars, and so on.

This is immediately useful for a company like Meta because it allows for object-level modifications across its content creation platforms. So if you took a video of someone dancing and you desperately wanted to give them a bobblehead, SAM 3, I'm told, would allow you to quickly do that. Other AI models, such as Gemini, can also segment, but supposedly the SAM models are better and more precise at this specific task.

Beyond bobblehead videos, the potential of this model seems enormous for real estate, cities, and, of course, many other things. Using the above image as an example, you can quickly imagine SAM 3 being used to count and track modal splits across a city, and then make planning decisions based on real-time data.

People are also using it for real estate purposes. Pair the model with satellite images, and you can ask it to tell you how many houses have a pool, which houses recently had their roof replaced (and have solar panels), how many cars are parked on a street, how many cars are parked at Canadian Tire, and the average building lot coverage in an area.

You could also use it to swap out finishes in a real estate listing (including in videos), and get material/area takeoffs ahead of a construction project. I don't know for sure, but I would also imagine that this model would make a great building condition inspector. Come to think of it, I'd love a SAM 3 that could walk our construction sites and document every little detail!

Of course, a lot of these use cases are already being tackled. But the models are getting that much better. And that will lead to even more innovation.

Cover photo by Above Horizon on Unsplash

Towards the end of last year, Meta released SAM 3, which stands for the third version of its Segment Anything Model. The way it generally works is that it allows you to detect, edit, and experiment with things in images and videos. For example, if you were looking at a video of a street, you could ask it to find all the scooters (which I did below), count the number of pedestrians wearing black pants, blur all the license plates on the cars, and so on.

This is immediately useful for a company like Meta because it allows for object-level modifications across its content creation platforms. So if you took a video of someone dancing and you desperately wanted to give them a bobblehead, SAM 3, I'm told, would allow you to quickly do that. Other AI models, such as Gemini, can also segment, but supposedly the SAM models are better and more precise at this specific task.

Beyond bobblehead videos, the potential of this model seems enormous for real estate, cities, and, of course, many other things. Using the above image as an example, you can quickly imagine SAM 3 being used to count and track modal splits across a city, and then make planning decisions based on real-time data.

People are also using it for real estate purposes. Pair the model with satellite images, and you can ask it to tell you how many houses have a pool, which houses recently had their roof replaced (and have solar panels), how many cars are parked on a street, how many cars are parked at Canadian Tire, and the average building lot coverage in an area.

You could also use it to swap out finishes in a real estate listing (including in videos), and get material/area takeoffs ahead of a construction project. I don't know for sure, but I would also imagine that this model would make a great building condition inspector. Come to think of it, I'd love a SAM 3 that could walk our construction sites and document every little detail!

Of course, a lot of these use cases are already being tackled. But the models are getting that much better. And that will lead to even more innovation.

Cover photo by Above Horizon on Unsplash