FFmpeg: The Incredible Technology Behind Video on the Internet | Lex Fridman Podcast #496

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep496-sa
See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.
0:00 – Episode highlight
2:17 – Introduction
5:35 – Weirdest things VLC opens
9:59 – How video playback works
19:20 – Video codecs and containers
30:07 – FFmpeg explained
51:07 – Linus Torvalds
55:46 – Turning down millions to keep VLC ad-free
1:10:04 – FFmpeg & Google drama
1:29:18 – FFmpeg developers
1:35:55 – VLC and FFmpeg
1:40:29 – History of FFmpeg
1:43:46 – Reverse engineering codecs
1:57:01 – FFmpeg testing
2:01:08 – Assembly code (handwritten)
2:25:26 – Rust programming language
2:34:42 – FFmpeg and Libav fork
2:43:04 – Open source burnout
2:50:51 – x264 and internet video
3:04:07 – Video compression basics
3:11:04 – CIA and fake VLC
3:21:39 – Ultra low latency streaming
3:39:07 – AV2 codec and video patents
3:48:59 – VLC backdoors
3:59:14 – Video archiving
4:05:51 – Future of FFmpeg and VLC

Transcript:
https://lexfridman.com/ffmpeg-transcript

CONTACT LEX:
Feedback – give feedback to Lex: https://lexfridman.com/survey
AMA – submit questions, videos or call-in: https://lexfridman.com/ama
Hiring – join our team: https://lexfridman.com/hiring
Other – other ways to get in touch: https://lexfridman.com/contact

EPISODE LINKS:
FFmpeg on X: https://x.com/FFmpeg
FFmpeg: https://ffmpeg.org/
VideoLAN (VLC): https://www.videolan.org/
VideoLAN on X: https://x.com/videolan
Jean-Baptiste's Website: https://jbkempf.com/
Jean-Baptiste's LinkedIn: https://www.linkedin.com/in/jbkempf/
Jean-Baptiste's GitHub: https://github.com/jbkempf
Kieran's X: https://x.com/kierank_
Kieran's LinkedIn: https://bit.ly/3OORhmC
Kieran's GitHub: https://github.com/kierank

SPONSORS:
To support this podcast, check out our sponsors & get discounts:
Larridin: Measure AI adoption in your business.
Go to https://lexfridman.com/s/larridin-ep496-sa
Blitzy: AI agent for large enterprise codebases.
Go to https://lexfridman.com/s/blitzy-ep496-sa
BetterHelp: Online therapy and counseling.
Go to https://lexfridman.com/s/betterhelp-ep496-sa
Fin: AI agent for customer service.
Go to https://lexfridman.com/s/fin-ep496-sa
LMNT: Zero-sugar electrolyte drink mix.
Go to https://lexfridman.com/s/lmnt-ep496-sa
Perplexity: AI-powered answer engine.
Go to https://lexfridman.com/s/perplexity-ep496-sa

I don even like Lex Fridman but this is the greatest and most inspiring podcast I have ever seen. I now want to program for FFmpeg in Assembly language.

I used to video mash i and p frame wrangling

📖 The Importance of Excellent Code Quality (0:00 – 7:16)
├─ 2:00 The Power of FFmpeg and VLC Technology
└─ 4:09 The Power of Open Source Infrastructure
📖 The Iconic VLC Logo and Its Significance (7:16 – 14:05)
└─ 10:14 The Inner Workings of Video Processing Technology
📖 Video Compression Techniques for Human Perception (14:05 – 19:58)
└─ 16:20 Video Compression Techniques and Mathematical Tricks
📖 Understanding Video Container Formats and Codecs (19:58 – 25:36)
└─ 22:45 Designing Resilient Video Players
📖 Video Compression Techniques and Redundancy (25:36 – 31:27)
└─ 28:37 The Inner Workings of Video Compression Technologies
📖 FFmpeg as a Multimedia Processing Toolbox (31:27 – 37:09)
├─ 33:56 FFmpeg's Democratization of Complex Video Technology
└─ 35:32 The Impact of FFmpeg on Global Communication
📖 The Power of Open Source Software Development (37:09 – 43:32)
├─ 39:07 Open Source License Challenges in FFmpeg
└─ 40:58 Open Source Licensing Explained
📖 Licensing Differences Between GPL and LGPL (43:32 – 48:49)
└─ 45:59 Open Source Licensing and Contribution Requirements
📖 Importance of Community Agreements in Open Source (48:49 – 54:28)
└─ 51:29 Maintaining High-Quality Open Source Codebases
📖 Maintaining Open Source Software Integrity (54:28 – 1:00:01)
└─ 57:03 Student-Run Campus Experiment and Its Success
📖 Early Video Streaming and Network Requirements (1:00:01 – 1:12:06)
├─ 1:03:43 The Origins of VLC Media Player
├─ 1:05:52 Maintaining Integrity in Software Development
└─ 1:08:36 The Open Source Movement and Volunteer Developers
📖 The Role of AI in Security and Open Source (1:12:06 – 1:20:55)
├─ 1:15:17 Incentives for Open Source Security Contributions
└─ 1:17:25 Misconceptions about Open Source Project Support
📖 Challenges of Collaboration with Large Companies (1:20:55 – 1:30:12)
├─ 1:24:03 Youth Contributions to Open-Source Video Technologies
└─ 1:28:06 The Power of Open Source Communities
📖 The Challenges of Programming in FFmpeg (1:30:12 – 1:35:56)
├─ 1:32:51 The Importance of Open Source Contributions
└─ 1:34:22 The Power of Passion Projects in Software Development
📖 FFmpeg and VLC's Coexistence and Interdependence (1:35:56 – 1:43:49)
├─ 1:37:52 The Role of VideoLAN in FFmpeg's Success
└─ 1:40:23 The Evolution of FFmpeg Video Codecs
📖 The Early Days of Video Codecs Development (1:43:49 – 1:50:24)
└─ 1:47:01 Reverse Engineering Video Compression Algorithms
📖 Debugging Video Codecs with Sample Videos (1:50:24 – 1:55:35)
└─ 1:52:36 Debugging Video Codecs and DRM Systems
📖 FFmpeg's Testing Process for Codec Integration (1:55:35 – 2:01:56)
└─ 1:58:50 Video Encoding Challenges and Format Conversions
📖 Challenges of Supporting Multiple Operating Systems (2:01:56 – 2:07:36)
├─ 2:03:58 The Art of Assembly in Video Processing
└─ 2:05:39 FFmpeg's Optimizations for Video Decoding
📖 Optimizing Video Decoding with Handwritten Assembly (2:07:36 – 2:14:31)
├─ 2:09:44 The Limits of Compiler Optimization
└─ 2:12:31 Optimizing Assembly Code for Multiple Operating Systems
📖 The Importance of CPU Architecture in Coding (2:14:31 – 2:21:35)
├─ 2:16:24 Optimizing Video Processing for Real-Time Performance
├─ 2:18:01 Importance of Assembly Language Programming
└─ 2:19:57 Reviving Assembly Language in Video Encoding
📖 The Challenges of Large Language Models (2:21:35 – 2:26:56)
└─ 2:23:19 Importance of Assembly Language in Modern Programming
📖 The Challenges of Integrating New Technologies (2:26:56 – 2:34:03)
└─ 2:30:14 The Challenges of Secure Assembly in Programming
📖 Complexity of Arm and x86 Architectures (2:34:03 – 2:39:48)
└─ 2:36:49 FFmpeg and Libav Split and Reunification
📖 Maintainer Burnout in Open Source Communities (2:39:48 – 2:46:51)
├─ 2:42:19 Challenges of Maintaining Open Source Projects
└─ 2:43:51 Celebrating Unsung Heroes of Open Source Development
📖 The Importance of Open Source Software (2:46:51 – 2:52:11)
├─ 2:48:34 The Importance of Making Complex Tech Accessible
└─ 2:50:22 The H.264 Video Standard and Its Impact
📖 Advances in Video Compression Techniques (2:52:11 – 2:58:17)
├─ 2:54:59 Optimizing Video Encoding for Quality
└─ 2:56:23 The Evolution of Video Encoding Technology
📖 Anime Video Encoding and Subtitle Challenges (2:58:17 – 3:04:06)
├─ 3:00:11 Video Compression Technologies Compared
└─ 3:01:49 Advances in Video Compression Technology
📖 Video Compression Techniques in FFmpeg (3:04:06 – 3:10:25)
├─ 3:06:04 Video Compression Techniques in FFmpeg
└─ 3:07:42 FFmpeg's Complexity and Parameter Tuning
📖 CIA's Custom VLC Plugin for Data Theft (3:10:25 – 3:16:24)
├─ 3:12:47 Security Risks of Open Source Software Downloads
└─ 3:14:27 VLC Security Risks and Malware Detection
📖 VLC Security and Process Isolation (3:16:24 – 3:21:47)
└─ 3:18:37 Challenges of Building a Secure Multimedia Player
📖 Challenges of Streaming Video Over Networks (3:21:47 – 3:27:25)
└─ 3:24:18 Video Encoding and Bitrate Complexity
📖 Low Latency Video Encoding for Real-Time Control (3:27:25 – 3:34:40)
└─ 3:31:24 Real-Time Video Streaming and Low Latency Technology
📖 Achieving Real-Time Video Processing Over the Internet (3:34:40 – 3:40:03)
├─ 3:36:50 Real-Time Machine Control and Open Source Licensing
└─ 3:38:18 Advancements in Video Compression Technology
📖 AV2 Video Codec Specification and Implementation (3:40:03 – 3:45:26)
├─ 3:41:45 Advancements in Video Compression Technology
└─ 3:43:28 Advantages of AV1 and AV2 Video Codecs
📖 Patent Issues in Video Compression (3:45:26 – 3:52:53)
├─ 3:47:48 Patent Issues in Video Compression Technology
└─ 3:50:14 Challenges of Entrepreneurship in Regulated Environments
📖 Software Security and Government Backdoors (3:52:53 – 3:59:14)
└─ 3:55:06 VLC's Limitations in Surveillance and Support
📖 Importance of FFmpeg in Digital Archiving (3:59:14 – 4:04:17)
└─ 4:00:51 Importance of Lossless Video Compression in Archiving
📖 Preserving Digital History and Media Archives (4:04:17 – 4:09:10)
└─ 4:06:35 Future of Multimedia and Brain-Computer Interfaces
📖 Haptic Technology in Media Players (4:09:10 – 4:18:21)
├─ 4:12:47 FFmpeg's Evolution and Impact on Technology
├─ 4:14:39 VLC's Widespread Use in Various Industries
└─ 4:16:14 The Importance of Not Regretting Mistakes

Europe is friendly to people and entrepreneurs. You distort reality. You're confusing it with the US which is in bed with entrepreneurs and against people.
Europe should not go the abusive US path.

Would love to know what they think about the MoshUp datamoshing app 🙂

I saw two traffic cones up the road, guess what they reminded me of? Guess it and make it a wild one at this rate

They are living legends. Thanks for pointing us to all the unsung heroes behind the scenes too

FFmpeg is better than adobe's proprietary media engine at encoding. I've found I get faster encoding, lower file sizes at better quality with open source ffmpeg than adobe's media encoder.

Great memories! When we started the Network2000 project in 1995/1996, I was the president of the VIA non-profit mentioned around 1:00:00.
I am very happy that this small project survived, thrived and became the VLC we all love.
Interested people can find our Network2000 project report in the link posted below (in French).
Thanks for the excellent interview.

Brilliantly amazing interview ❤
Please get Fabrice Bellard next. He is a superstar.

Holy shit that compilation strategy o.0

Thank you Lex! I really appreciate what you doing! What a nice episode and I loved how humble are your guest

LEX LEXX!!! Now is time to have…

Super interesting conversation. So excited too about the plans for expanding the multimedia concept too when it comes to standardised format. Like a video format for directly recording into 4D gaussian splatting, perhaps also from lightfield cameras or multiple cameras is a fixed setup that could require serious compression.

Hey Lex, where is podcast with Khabib?

59:22 Finally a networking technology when you don't have Brouteurs :')

I can't simply listen to this if there are walls of interesting text every 25 seconds. 🙁

Shortest Lex clip

Did he ever mentions when the AI subs are coming?

whenever you put a new computer up, you install two things. winrar & vlc.

It's been 3 years but I'm back in my lex era. This is more like the content I like. Just my taste though. All love

Thank you for the great content!