Matt says that WordPress itself was once a fork from another project. When enough contributors choose a new direction and get behind that project, the project takes flight. If, for example, 10% of the WP developers were to leave his project and start contributing somewhere else, he'd be worried.