Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.
|
Forum Index : Microcontroller and PC projects : LINUX: Filesystem oddness....
Page 1 of 2 | |||||
Author | Message | ||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
4TB WD Red drive, ext4. 76.9GB free. Windows insists the drive has ZERO bytes available. I tried deleting some old files on it, Windoze STILL says there are zero bytes available. Linux reports 76.9GB free UNTIL I unmount, then remount, then even Linux says there is zero bytes available. Getting suspicious at this point, at terminal: sudo fsck /dev/sdc1 Report comes back as "Clean". I can access the files just fine in both Linux locally, or via the network in Windows, but if I try to copy anything to it, it moans(in windows) that the drive is full, which it is not. In Linux, I can still copy files to the drive, EVEN THOUGH Linux is ALSO reporting zero space. This is very odd. Does anyone know a more advanced fsck command I can run to perhaps do a more in-depth file system check? The one I run from the terminal completes in about three seconds - much faster then the old ext2/ext3 fsck checks which took ages. Photos: The next thing I will do, is move some files off this drive - it is quite full. How many GB would ext4 reserve for FS maintenance on a 4TB drive? Perhaps this is just a case of the drive being too full? Smoke makes things work. When the smoke gets out, it stops! |
||||
Turbo46 Guru Joined: 24/12/2017 Location: AustraliaPosts: 1584 |
Are the deleted files going to the recycle bin on that drive? That may cause confusion. I'd try emptying it. Bill Keep safe. Live long and prosper. |
||||
SimpleSafeName Senior Member Joined: 28/07/2019 Location: United StatesPosts: 284 |
How are you reading/writing an ext4 drive in Windows? The new Linux subsystem? I did notice that you have an older version of fsck. Mine is 2.34 As for nodes, or whatever ext4 uses, that should be invisible to you I would think. Is it just sdc3 giving you trouble? |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
@ Turbo46: Recycle bin is empty. Makes no difference. @ SimpleSafeName: The ext4 drives are on a separate Linux-based computer, sitting on the network and sharing with Windoze machines via SAMBA2. The drives are mapped to the Windoze filesystem tree - Windoze does not care about what format the drives are, as the host at the other end of the network connection takes care of all that. Yes, just sdc1. I can write to all the others, as there is plenty of space on them. Smoke makes things work. When the smoke gets out, it stops! |
||||
Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 5648 |
Last time I played with a mix of linux and windows (a while ago, I admit), windows refused to see ext4 at all. A fully formatted ext4 drive always came back as having no space left according to windows, IIRC, and it just refused to do anything with it at all. Of course, that may have changed now. I *think* the system space required on the drive depends on the number of nodes, not the amount of data. So if you save a lot of short files you'll run out of node space before you run out of data space. Or something like that. I'm very rusty on this stuff now. IIRC you can tell ext4 to increase the number of nodes (inodes?) available but I can't for the life of me remember how now. Edited 2021-09-18 16:41 by Mixtel90 Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
Volhout Guru Joined: 05/03/2018 Location: NetherlandsPosts: 3422 |
Ive seen something similar before, that drives simply refuse to be filled up to the last byte. I associated this with the fact that high density drives are lossy. Not only ssd, but also harddrives. And the os on the drive itself reserves space to swap blocks in case one gets bad. In the ATA config you can, with tools from the drive manufacturer, see these reservations, and even when the os decides to swap blocks. Not saying this is the case in your situation, but it may be. 2 percent reservation may be totally valid. Edited 2021-09-19 01:41 by Volhout PicomiteVGA PETSCII ROBOTS |
||||
SimpleSafeName Senior Member Joined: 28/07/2019 Location: United StatesPosts: 284 |
"@ SimpleSafeName: The ext4 drives are on a separate Linux-based computer, sitting on the network and sharing with Windoze machines via SAMBA2. The drives are mapped to the Windoze filesystem tree - Windoze does not care about what format the drives are, as the host at the other end of the network connection takes care of all that. Yes, just sdc1. I can write to all the others, as there is plenty of space on them." Ahh, for some reason I was thinking that you were checking/repairing the drive from Windows. That would be a neat/inadvisable trick... And I'm familiar with SAMBA, not much use to me since I use a standalone machine at home and sneakernet works just fine for me. :) For my home use I usually just hang out in Linux. Any required access to Windows files is just a click away. And if I need to run a Windows program in Linux I use VirtualBox. That's how I have one of my work laptops set up. Works great. In any case, you can increase the number of inodes in EXT4 if you like: https://askubuntu.com/questions/600159/how-can-i-create-an-ext4-partition-with-an-extra-large-number-of-inodes#:~:text=See%20the%20man%20page%20for%20mkfs.ext4.%20Option%20-N,fit%20your%20situation%20when%20you%20create%20the%20filesystem. |
||||
JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3641 |
Is this a drive you've used for quite a while and been OK? I ask because I've met "drives" which are fake :( Odd. Very. Please try sudo e2fsck -f /dev/sdc1 I think your existing command is seeing the drive is "clean" so doing nothing useful. John |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
@ SimpleSafeName: You can get a downloadable Linux filesystem driver thing for Windoze, which give Windoze full access to Linux drives if you were to plop one in a Windoze machine. I forget the name, but I know you can get a 3rd-party application that will add Linux filesystem support to Windows so Windoze can read/write to them just like any other Windoze drive. @ John: Thanks for the expanded fsck thing - I will run that now. Yes, I've had this drive since brand new, never gave any issues before, and definitely purchased a genuine one from a reputable OEM seller. I've never met hard-drives that were fake of any brand, but I do know of the fake USB drives that are programmed to report the wrong capacity. EDIT: Here are the results.... At this point, I plan to move some files off that drive to free up about 100GB or so, and then see if Windoze reports any space or not, and also see if anything changes as far as Linux is concerned. Linux seems perfectly happy with the drive for the moment. I could try rebuilding the Puppy Linux boot USB I guess. I DID add one more drive to this system, which became sdd1, but that should not have upset anything I would not have thought.... Edited 2021-09-19 16:01 by Grogster Smoke makes things work. When the smoke gets out, it stops! |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
I've just discovered that in Linux, TWO different applications are reporting the change in capacity. BOTH of them report the same, if you run them one after the other, but side-by-side, they show the difference. pMount and Partview. This now explains the difference in Linux, as two different applications seem to be calculating the free space differently. Is there a command I can run from the console, to get a report on what the Kernel thinks the free space is rather then via a GUI application? Smoke makes things work. When the smoke gets out, it stops! |
||||
Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 5648 |
Anything here of interest? https://www.binarytides.com/linux-command-check-disk-partitions/ https://phoenixnap.com.br/kb/linux-check-disk-space Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
thwill Guru Joined: 16/09/2019 Location: United KingdomPosts: 3807 |
Yup, it's called 'df': https://www.tecmint.com/how-to-check-disk-space-in-linux/ Best wishes, Tom Game*Mite, CMM2 Welcome Tape, Creaky old text adventures |
||||
JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3641 |
hmm, I'm sorry to say I don't know what's wrong (if anything). df has some options (man df will explain) such as df -h which prints in "human readable" units (GB etc). When I do mount | grep ext4 I get just my mounted ext4 partitions e.g. /dev/sda5 on / type ext4 (rw,noatime,data=ordered) /dev/sda7 on /home type ext4 (rw,noatime,data=ordered) I guess you get something similar. John |
||||
Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 5648 |
I've no idea how to attack this. At one time I'd have set up a temporary box booting from a live filesystem (Mint) and adding this as the only HDD. Possibly a case of back the lot up to another drive and prepare to wipe and reformat in case it's something dodgy with the partition table or something. If you do *anything* to the drive your data's at risk if it's something like that, either immediately or in the future. And I've *finally* got round to putting a mouse socket on my CMM2. :) Edited 2021-09-20 00:45 by Mixtel90 Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
robert.rozee Guru Joined: 31/12/2012 Location: New ZealandPosts: 2282 |
3640 / 76.8 = 48 (approximately) 1024 / 24 = 43 (approximately) makes me wonder if you may have simply found a small bug where someone is using 1000 instead of 1024 when calculating file sizes. also see: https://en.wikipedia.org/wiki/Ext4 and https://en.wikipedia.org/wiki/Journaling_file_system with ext4, there are journaling structures on the disk that will be consuming a variable quantity of space. i've often seen drives being reported by the operating system as having zero bytes free when the O/S wishes to signal that the drive is being treated as read-only for 'whatever' reason. cheers, rob :-) |
||||
phil99 Guru Joined: 11/02/2018 Location: AustraliaPosts: 1721 |
Some time ago the "Metric Authorities" decreed that the multipliers should only be based on 10^3 and 2^10 should not be used. Drive manufacturers like this as their drives look bigger. Software writers still use both, causing some confusion. I don't know how ext4 works but in some software the unusable space between the end of a file and the end of its allocation unit is shown as free. Reformatting with smaller allocation units can recover some of that space, if there are a lot of small files. |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
Results of sudo df -h: This is ALSO reporting zero available. I think I will do Mick's suggestion - copy everything off of this drive for the moment, reformat it, and copy everything back again. Pit of a PITA, but I am suspicious of a drive reporting zero available, when I know there is more space there, and Linux even lets me copy files to the drive - despite some of Linux's own tools reporting zero space. I suppose one thing I could try first, is to boot up a USB version of Mint, and see if it also is reporting the same kind of oddness, then proceed from there. Smoke makes things work. When the smoke gets out, it stops! |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
Mint also reports zero - something odd going on here, so I will now proceed to remove everything from that drive, reformat it, and copy everything back. Smoke makes things work. When the smoke gets out, it stops! |
||||
JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3641 |
It may depend which user is doing the enquiry/copy and which fs (file system) kind (here, ext4). If root, you can use every byte (not always wise as some file systems get slow as they get extremely full/fragmented). If non-root, there is typically a threshold where the fs is shown as full to reduce/prevent the problem. Rebuilding by copying everything will reduce fragmentation but if the issue is along the lines just mentioned will make no difference. John |
||||
Grogster Admin Group Joined: 31/12/2012 Location: New ZealandPosts: 9030 |
Good point. I will MOVE 100GB or so off that drive to one of the other ones, and see what happens. Smoke makes things work. When the smoke gets out, it stops! |
||||
Page 1 of 2 |
Print this page |