Magento 2 missing setup folder

While working in a local dev environment using Docker, I’ve had Magento 2 missing setup folder from time to time, on environments using Warden or SwiftOtter/Den. Also found reports that it happens using ddev.

The most common situation is when switching between branches having different magento2 versions, therefore having different magento2-base module versions. This is the module responsible for mapping some folders structure, which can be seen in it’s own composer.json file, vendor/magento/magento2-base/composer.json at node "extra" > "map".

Based on the deploy strategy, it would copy or symlink the mapped folders.

The most common reason for missing setup folder is that some files that magento2-base is mapping are already in place and your magento2 root composer.json is missing the `”extra”: { “magento-force”:”override” } node. If this has been ruled out, you can continue reading.

What happens is that the sync between guest and host fails in some unpredictable modes, causing weird file system errors like Resource busy or reporting a folder to not exist, but when trying to create it, it reports it already exists.

Once it throws an error, further files/folders mapping is interrupted, and this is how you get the more visible situation of missing setup folder.

Example:

m246ce-php-fpm:/var/www/html$ ls /var/www/html/./pub/media
ls: /var/www/html/./pub/media: No such file or directory

m246ce-php-fpm:/var/www/html$ mkdir /var/www/html/./pub/media
mkdir: can't create directory '/var/www/html/./pub/media': File exists

m246ce-php-fpm:/var/www/html$ ls /var/www/html/pub/media
ls: /var/www/html/pub/media: No such file or directory

m246ce-php-fpm:/var/www/html$ mkdir /var/www/html/pub/media
mkdir: can't create directory '/var/www/html/pub/media': File exists

Creating and removing pub/media folder from host machine works, and checking guest afterward reveals sync issues with a “resource busy” error:

➜  m246ce git:(246ce) ✗ den shell
m246ce-php-fpm:/var/www/html$ ls pub/
cron.php          errors            get.php           health_check.php  index.php         media             test

m246ce-php-fpm:/var/www/html$ rm -rf pub/media/
rm: can't remove 'pub/media': Resource busy

Checking mutagen sync list command shows it as running without issues:

--------------------------------------------------------------------------------
Identifier: sync_Ne5nDF94KNd5fnYrDwCowEdvX0hkkXiYLb4Bb1iQGp5
Alpha:
	URL: /Users/mm/m246ce
	Connected: Yes
	Synchronizable contents:
		29590 directories
		77632 files (495 MB)
		0 symbolic links
Beta:
	URL: docker://de9a469a3b3ebcabc3e3ea5a0ef5c2dfdd91939eb8b86bb433ec3dd050ae6abc/var/www/html
	Connected: Yes
	Synchronizable contents:
		29590 directories
		77632 files (495 MB)
		0 symbolic links
Status: Watching for changes

mutagen version 0.16.4

Unfortunately, I haven’t made a note on all versions of the implied stack and the issue disappeared after I upgraded Docker. It’s working fine now with Docker Engine v23.0.5, Compose v2.17.3, Desktop v4.19.0.

My conclusion is it is caused by a specific state of files in the folder in a combination with host/guest/mutagen/docker versions, which can be hard to debug.

If upgrading does not solve it for you, then you can go further debugging into magento2-base module.

To debug it, you can do these:

rm -rf vendor/magento/magento2-base folder, do it from the env shell, not from the host machine, (to avoid any sync issues like mutagen sync which might not trigger a sync in time for large amount of files), then follow with composer install -vvv. Check the output in detail, and you might see something like I’ve encountered for example:

> post-install-cmd: MagentoHackathon\Composer\Magento\Plugin->onNewCodeEvent
start magento deploy via deployManager
start magento deploy for magento/magento2-base
copy(/var/www/html/./pub/media/.htaccess): Failed to open stream: No such file or directory
jump over deployLibraries as no Magento libraryPath is set

If shown information is not useful, you can hack into magento-composer-install module and dump/echo stuff, like mentioned here for ignored destination cases: https://magento.stackexchange.com/a/350046/7746 , hoping to find cause of it.

Here’s an output of my debugging session, that made me realise I might be dealing with weird filesystem syncing issues:

Magento 2 local development environment orchestrators:

https://github.com/swiftotter/den
https://github.com/wardenenv/warden
https://github.com/markshust/docker-magento
https://github.com/ddev/

Magento 2 missing setup folder

Related links for Magento2 setup issues:

Magento 2 local development environment orchestrators:

Related posts