Home
JAQForum Ver 20.06
Log In or Join  
Active Topics
Local Time 23:49 03 Dec 2021 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : LINUX: How do you LONG-FORMAT a volume?

     Page 1 of 2    
Author Message
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 06:35am 24 Oct 2021
Copy link to clipboard 
Print this post

Howdy.

I have a 1TB USB3 drive which is suspect.
Every time I write to it, at about the 600GB mark, the write speed slows to 1MB per second instead of the usual 100MB or so per second for USB3.

Placing the drive up to my ear, I can hear that the heads are moving all over the place, so I think I have bad-sectors on this drive.

The problem is, with all newer OS's, they only seem to give you the choice of a quick-format, not a long one.  That is perfectly understandable given the size of the average modern HDD, but there are times - like this, when I need it.

Windoze DOES allow a long-format option by simply unticking the "Quick format" checkbox, but I want to use this USB drive in different systems, so I want to format it to FAT32, but Windoze won't let me do that - it is NTFS or exFAT - that's it.
Windoze won't allow me to format a 1TB drive as FAT32.

So, can someone point me in the direction of a Linux application that can do long-formatting?
I don't really care about how long it will take to do this - I will set it going and then go to bed and check on it the next day.  Whatever software would need to be able to identify bad sectors and mark them as such, and then just keep on going with the format.  You would end up with LESS then the full capacity, but that's fine.

This is only a drive I use to move files from one system to another, so I don't care about losing some capacity etc, only that it is compatible with all systems - which is why I am using FAT32, or wanting to, and WOULD slow-format it in Windoze if Windoze would let me, but it won't.  
Smoke makes things work. When the smoke gets out, it stops!
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 07:06am 24 Oct 2021
Copy link to clipboard 
Print this post

Perhaps there is a command-line/console method?
That would not surprise me.
Linux being Linux.
Smoke makes things work. When the smoke gets out, it stops!
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 1196
Posted: 07:48am 24 Oct 2021
Copy link to clipboard 
Print this post

You know that you can't put files over 4GB on a FAT32 drive, don't you? That might be a bit restrictive.

You can format FAT32 in Win10 using Powershell.
https://www.maketecheasier.com/format-drive-fat32-windows10/

There's a GUI utility called "disks" that comes with Ubuntu systems that can do FAT32.
https://help.ubuntu.com/stable/ubuntu-help/disk-partitions.html.en
Edited 2021-10-24 17:54 by Mixtel90
-- Mick

Zilog Inside! nascom.info for Nascom & Gemini
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 08:07am 24 Oct 2021
Copy link to clipboard 
Print this post

Yes, fully aware of the 4GB file limit of FAT32.
Most files moved are way less then 1GB on this particular drive.

EDIT: "Disks" in the likes of Mint - can it LONG FORMAT, or just more of the quick-format?

I need to now boot up one of my Mint machines and check!
Edited 2021-10-24 18:08 by Grogster
Smoke makes things work. When the smoke gets out, it stops!
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 1196
Posted: 08:29am 24 Oct 2021
Copy link to clipboard 
Print this post

You could use DBAN or something to totally delete everything from the drive. DBAN also repeatedly writes to all sectors to restore the drive to "as new" condition. Any format should work after that.

Just don't let your enemies get hold of DBAN. Be *absolutely certain* that it's pointed at the correct drive. It's dangerous. :)
-- Mick

Zilog Inside! nascom.info for Nascom & Gemini
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 2534
Posted: 02:08pm 24 Oct 2021
Copy link to clipboard 
Print this post

I suspect most OS utilities do not do an actual format (in the sense it used to be, i.e. writing headers, trailers, timing markers, etc, per sector) but perhaps do at least write to every block (and probably read the block back) - an action which with modern disks should result in bad blocks being remapped.

There are SMART short & long tests and the like, which at least provide some info on a disk's health, e.g. (on Linux)

smartctl -t short /dev/sdb

smartctl -l selftest /dev/sdb

The whole smartctl command set is quite interesting (er, I may be abusing that word though) .

When I get a used disk I start with

smartctl -a /dev/sdb

One of the things I look at is the Reallocated_Sector_Ct

(In all cases, replace /dev/sdb with your disk's path.)

John
Edited 2021-10-25 00:10 by JohnS
 
LeoNicolas

Senior Member

Joined: 07/10/2020
Location: Canada
Posts: 212
Posted: 02:32pm 24 Oct 2021
Copy link to clipboard 
Print this post

Don't expect windowed apps for all advance features. The most part of them are available on the command line or syscals or in an application that is not default on the Linux distribution you are using.
Advanced users usually use command line for advanced actions. Why? Because we can easily create scripts to perform complex things... And when a new feature is added, it's done in the kernel and is created a command line for it.

You can try to use gparted to reconstruct you disk partitions and reformat it
Edited 2021-10-25 00:36 by LeoNicolas
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 2534
Posted: 04:46pm 24 Oct 2021
Copy link to clipboard 
Print this post

BTW, if smartctl longtest shows problems I have successfully corrected them by writing (any pattern but /dev/zero is easy) to the sector(s) reported.

It makes the drive "spare" (replace) the sector(s).

You lose the data in the bad block(s) of course.

John
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 12:32am 25 Oct 2021
Copy link to clipboard 
Print this post

Lovely, thanks chums, I will try some of those and report back with the results of the test.
Smoke makes things work. When the smoke gets out, it stops!
 
scruss
Newbie

Joined: 20/09/2021
Location: Canada
Posts: 34
Posted: 02:14am 25 Oct 2021
Copy link to clipboard 
Print this post

If you want to do a really long formatting/checking, run:
fsck -vcck /dev/device-PARTITION

This runs the `badblocks` utility, and will mark/remap bad sectors. This is in addition to SMART remapping. Really dedicated types will run badblocks for days over their new or suspect drives to ensure that SMART isn't hiding problems that will show up later.
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 06:30am 25 Oct 2021
Copy link to clipboard 
Print this post

"Permission denied"  





I tried using Puppy first, but it did not have the command, so I tried Mint, but it didn't have it either!

so, learning a LITTLE more about the Linux terminal each time I use it, I installed the package:

sudo apt install smartmontools [OK]
smartctl -t short /dev/sdb1


DENIED!!!!  

@ scruss: Not sure why I never thought of fsck myself!  I should have!  Thanks.  Will try that.  Possibly cos in the early days of my Linux experience when ext2 was still the in-thing filesystem, fsck took so long to do a check of an unclean volume, that I replaced the "s" with a "u" in that command.....as I think, did many people of the time!    ext4 is my default now, and it does not have those long fsck wait times, which is brilliant.
Smoke makes things work. When the smoke gets out, it stops!
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 06:43am 25 Oct 2021
Copy link to clipboard 
Print this post

fsck seems happier then smartctl:





However, this completed in about three seconds.  Is it still running in the background?

scruss?

The activity LED on the 1TB USB3 drive blinks about once every ten seconds or so, so I am wondering if this is a backgrounded thing?
Smoke makes things work. When the smoke gets out, it stops!
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 06:44am 25 Oct 2021
Copy link to clipboard 
Print this post

The thought just occurs that my smartctl attempts might have been getting denied as I did not run the command as sudo....

Yes?
No?
Smoke makes things work. When the smoke gets out, it stops!
 
Volhout
Guru

Joined: 05/03/2018
Location: Netherlands
Posts: 952
Posted: 10:05am 25 Oct 2021
Copy link to clipboard 
Print this post

In linux it is not allowed to modify the mounted system drive. Better run these tools while running from a usb live distro, or from a second harddrive.
If nothing goes right ... turn left
 
JohnS
Guru

Joined: 18/11/2011
Location: United Kingdom
Posts: 2534
Posted: 11:22am 25 Oct 2021
Copy link to clipboard 
Print this post

Yes, you'll need to run smartctl as root

It can wipe a drive of all data so is not by default runnable by a non-root user.

John
 
scruss
Newbie

Joined: 20/09/2021
Location: Canada
Posts: 34
Posted: 02:45pm 25 Oct 2021
Copy link to clipboard 
Print this post

Not sure if badblocks runs on anything but ext filesystems. Looks like fsck.fat didn't even try.

ext4 is basically the same as ext2, but with journaling. ext2 tools used to be slow because, before SMART was everywhere, they assumed you had to check everything, Now many of the tools check SMART, see it's fine, and skip real checks.
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 11:34pm 25 Oct 2021
Copy link to clipboard 
Print this post

@ scruss: OK.

I've played with smartctl again, and it is now moaning about an unsupported field in the command, even when I use their own example from the help:





I have tried other variations of this, but they all report the same thing.  For example: smartctl -t long /dev/sdb1 as in the example in JohnS's post above, but that also generates the same error.
Smoke makes things work. When the smoke gets out, it stops!
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 1786
Posted: 01:37am 26 Oct 2021
Copy link to clipboard 
Print this post

I don't know all the circumstances, but does it need to be /dev/sdb instead of /dev/sdb1?
 
Grogster

Guru

Joined: 31/12/2012
Location: New Zealand
Posts: 7750
Posted: 03:51am 26 Oct 2021
Copy link to clipboard 
Print this post

Nice idea.  Mint assigns the drive the path of /dev/sdb1, so that is what I used, but I will try leaving off the number....

Bugger me!
You were quite right.
I was using the Mint path, as I thought that was what the path was or is.



Smoke makes things work. When the smoke gets out, it stops!
 
phil99
Senior Member

Joined: 11/02/2018
Location: Australia
Posts: 227
Posted: 05:33am 26 Oct 2021
Copy link to clipboard 
Print this post

/dev/sdb1 is the partition path - allows fs checking only.
/dev/sdb is the disk hardware for bad sector checking.
 
     Page 1 of 2    
Print this page
© JAQ Software 2021