From mbmiller+l at gmail.com Wed Sep 2 17:47:54 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 2 Sep 2015 17:47:54 -0500 (CDT) Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") In-Reply-To: References: <03d18db2b12b27809a0daa0bcbad7b23.squirrel@www.ron-l-j.com> Message-ID: I just discovered that I've been receiving a message from "mdadm monitoring" every morning at about 8:00 am since July 2012, but these messages were delivered to root and then went to "nobody" because I never set a root email address in /etc/aliases. Anyway, it looks like these messages are all the same and they look like this: Date: Wed, 2 Sep 2015 07:55:39 From: mdadm monitoring To: root at taxa1 Subject: SparesMissing event on /dev/md/1:taxa1 This is an automatically generated mail message from mdadm running on taxa1 A SparesMissing event had been detected on md device /dev/md/1. Faithfully yours, etc. P.S. The /proc/mdstat file currently contains the following: Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sda3[0] sdb3[1] 2896392511 blocks super 1.2 [2/2] [UU] md0 : active raid1 sda2[0] sdb2[1] 33553336 blocks super 1.2 [2/2] [UU] unused devices: ---------end of message on previous line------------- Maybe this isn't telling me anything I didn't already know -- the raid doesn't boot normally and probably only one of the two drives has grub properly installed. Mike On Mon, 31 Aug 2015, Mike Miller wrote: > Despite all the help I had, way back in October 2012, I still never fixed > that RAID1. At this point I think I know what went wrong. I would like to > try to fix it so that I can upgrade the Ubuntu LTS version -- I'm still using > 12.04.5 LTS. > > See the output below from df, parted -l, /proc/mdstat and /etc/fstab. There > is a /boot directory, but that is in the / partition. The /boot partition is > not mounted and it isn't in the RAID1. > > I have been doing software updates, and kernels have been installed, but the > only one I can boot from is the one I installed originally. I assume that > since the /boot partition isn't mounted, the dozen or so kernels I see are in > the /boot directory in the / partition, and there are 341 MB of files in > there now. > > I'm hoping there is a way to get the appropriate files into the /boot > partition, get that mounted, get rid of the /boot directory in the / > partition and have the /boot partition mirrored on the two drives. > > Is there hope? Do any of you know how I should proceed? > > Mike > > > $ df -HT > Filesystem Type Size Used Avail Use% Mounted on > /dev/md1 ext4 3.0T 296G 2.6T 11% / > udev devtmpfs 8.4G 13k 8.4G 1% /dev > tmpfs tmpfs 3.4G 906k 3.4G 1% /run > none tmpfs 5.3M 0 5.3M 0% /run/lock > none tmpfs 8.4G 52M 8.4G 1% /run/shm > > > $ sudo parted -l > > Model: ATA ST3000DM001-9YN1 (scsi) > Disk /dev/sda: 3001GB > Sector size (logical/physical): 512B/4096B > Partition Table: gpt > > Number Start End Size File system Name Flags > 1 10.5MB 220MB 210MB /boot bios_grub > 2 220MB 34.6GB 34.4GB swap raid > 3 34.6GB 3000GB 2966GB / raid > > > Model: ATA ST3000DM001-9YN1 (scsi) > Disk /dev/sdb: 3001GB > Sector size (logical/physical): 512B/4096B > Partition Table: gpt > > Number Start End Size File system Name Flags > 1 10.5MB 220MB 210MB /boot bios_grub > 2 220MB 34.6GB 34.4GB swap raid > 3 34.6GB 3000GB 2966GB / raid > > > Model: Linux Software RAID Array (md) > Disk /dev/md0: 34.4GB > Sector size (logical/physical): 512B/4096B > Partition Table: loop > > Number Start End Size File system Flags > 1 0.00B 34.4GB 34.4GB linux-swap(v1) > > > Model: Linux Software RAID Array (md) > Disk /dev/md1: 2966GB > Sector size (logical/physical): 512B/4096B > Partition Table: loop > > Number Start End Size File system Flags > 1 0.00B 2966GB 2966GB ext4 > > > > $ cat /proc/mdstat > Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] > [raid10] > md1 : active raid1 sda3[0] sdb3[1] > 2896392511 blocks super 1.2 [2/2] [UU] > > md0 : active raid1 sda2[0] sdb2[1] > 33553336 blocks super 1.2 [2/2] [UU] > > unused devices: > > > > # /etc/fstab: static file system information. > # > # Use 'blkid' to print the universally unique identifier for a > # device; this may be used with UUID= as a more robust way to name devices > # that works even if disks are added and removed. See fstab(5). > # > # > proc /proc proc nodev,noexec,nosuid 0 0 > # / was on /dev/md1 during installation > UUID=f8a6d38d-3dd0-465e-b622-1cffdde52c1b / ext4 > errors=remount-ro 0 1 > # swap was on /dev/md0 during installation > UUID=f8be4924-e4a8-20ba-f284-3a90581a5322 none swap sw > 0 0 > > > > On Fri, 5 Oct 2012, Mike Miller wrote: > >> On Fri, 5 Oct 2012, ron at ron-l-j.com wrote: >> >>> The first thing I would try is booting into a shell and running fsck on >>> your boot disk. >>> Reboot the machine and press e in the grub screen for edit. >>> Go to the kernel line and press e to edit that line. >>> Then add to the kernel line >>> >>> init=/bin/bash >>> >>> when you boot you go straight to the shell. >>> then do a file system check on your boot drive >>> fsck -t ext3 /dev/sdax >>> t is for file system type in this case ext3 >>> and the x in sdax is your partition number. >>> sda is for sata drives, if you have an ide drive use hdax >>> >>> Most of the time its a file system error and was very common when I was >>> doing raid arrays in my advanced linux classes. Bad super block is also >>> common. >>> If you are still having troubles run fsck from a live cd/usb. But you will >>> have to run the cd in live mode, mount the offending drive, and chroot >>> into the drive. >>> As a last resort you can update-grub from the live cd after you have >>> chroot'ed into you installed environment. >>> Let me know what you encounter. >> >> >> Thanks, Ron. (I'm cc'ing you in case you are doing digests only and want >> to see this before the digest comes in.) >> >> Did you see that I was able to boot up to the window manager login prompt >> using an earlier version of the kernel? Can I just take it from there? >> >> My impression right now is that the problem was caused by my failure to >> reboot for weeks after installing some packages that required rebooting. >> Then I installed 400 more packages before rebooting. Is that possibly the >> cause of my troubles? Could it be that I just need to fix the packages and >> reboot? >> >> The other issue is that I have a RAID1, so mirrored drives, and I think >> that means I don't want to fsck them one at a time. df shows this: >> >> $ sudo df -HT >> Filesystem Type Size Used Avail Use% Mounted on >> /dev/md1 ext4 3.0T 220G 2.6T 8% / >> udev devtmpfs 8.4G 13k 8.4G 1% /dev >> tmpfs tmpfs 3.4G 832k 3.4G 1% /run >> none tmpfs 5.3M 0 5.3M 0% /run/lock >> none tmpfs 8.4G 148k 8.4G 1% /run/shm >> >> Does that mean I would fsck /dev/md1? >> >> Thanks again. I really appreciate your taking the time to reply. >> >> (One thing I've learned -- I should run package-manager daily and try to >> reboot soon when it is required. I often have a lot of stuff running that >> I don't want to kill, so reboots are a hassle.) >> >> Mike >> > From mbmiller+l at gmail.com Wed Sep 2 23:28:24 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 2 Sep 2015 23:28:24 -0500 (CDT) Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") In-Reply-To: References: <03d18db2b12b27809a0daa0bcbad7b23.squirrel@www.ron-l-j.com> Message-ID: Does this sound like a reasonable plan? After backing up all files and booting to a Live CD: (1) mount /dev/sda and /dev/sdb (2) Remove /boot directory from / partition on both sda and sdb (3) Install grub in /boot parttions on both drives. (4) mount /boot partitons of both sda and sdb (5) copy known-working kernel and two newest installed kernels to /boot partition from backup media (formerly in /boot of / partition) (6) make sure /boot partition is mounted in fstab (7) /boot partition should also be in the RAID1 Does that sound like it would fix it? Any ideas? More info below, if you're interested. ;-) I think my problem is related to this: http://unix.stackexchange.com/questions/3623/installing-grub-2-on-mdadm-which-device?rq=1 I'm pretty sure I installed grub on only one of the two drives. Then I didn't mount the /boot (I don't even know how to do that!). So now I have Ubuntu installed so that there is a /boot directory in the / partition, and there is a /boot partition. The / partition (and thus /boot *directory*) is in the RAID1, but the /boot partition is not in the RAID1. /dev/sda and /dev/sdb are identical and both show this with parted -l: Sector size (logical/physical): 512B/4096B Partition Table: gpt Number Start End Size File system Name Flags 1 10.5MB 220MB 210MB /boot bios_grub 2 220MB 34.6GB 34.4GB swap raid 3 34.6GB 3000GB 2966GB / raid df doesn't show a /boot partition... Filesystem Type Size Used Avail Use% Mounted on /dev/md1 ext4 3.0T 296G 2.6T 11% / udev devtmpfs 8.4G 13k 8.4G 1% /dev tmpfs tmpfs 3.4G 902k 3.4G 1% /run none tmpfs 5.3M 0 5.3M 0% /run/lock none tmpfs 8.4G 29M 8.4G 1% /run/shm /dev/sde1 fuseblk 1.6T 947G 554G 64% /media/New Volume /dev/sdd1 fuseblk 2.1T 319G 1.7T 16% /media/HD-CLU2 ...and the /boot *directory* in / is already using about 50% more space than /boot partition would allow: $ du -sm /boot 341 /boot $ df -HT /boot Filesystem Type Size Used Avail Use% Mounted on /dev/md1 ext4 3.0T 296G 2.6T 11% / $ \ls -1v /boot | cut -d'-' -f-2 | uniq -c 15 System.map-3.2.0 15 abi-3.2.0 15 config-3.2.0 1 grub 15 initrd.img-3.2.0 1 memtest86+.bin 1 memtest86+_multiboot.bin 15 vmlinuz-3.2.0 So it looks like Ubuntu updates have installed 15 kernels in the /boot directory in /. The big files are the ones beginning with initrd.img at about 14.6 MB apiece. The grub directory takes up only 5 MB. Best, Mike From mbmiller+l at gmail.com Thu Sep 3 01:10:24 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Thu, 3 Sep 2015 01:10:24 -0500 (CDT) Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") In-Reply-To: References: <03d18db2b12b27809a0daa0bcbad7b23.squirrel@www.ron-l-j.com> Message-ID: I meant to add that the problem of old kernels filling the /boot partition is a known issue: http://ubuntuforums.org/showthread.php?t=2238152 With Ubuntu 13.10 and later, we can use this command... sudo apt-get autoremove ...which (the page says) "will now removes those older kernels (images and header and related files too), leaving the current kernel and also 1 latter kernel." Seems reasonable. Mike On Wed, 2 Sep 2015, Mike Miller wrote: > Does this sound like a reasonable plan? After backing up all files and > booting to a Live CD: > > (1) mount /dev/sda and /dev/sdb > (2) Remove /boot directory from / partition on both sda and sdb > (3) Install grub in /boot parttions on both drives. > (4) mount /boot partitons of both sda and sdb > (5) copy known-working kernel and two newest installed kernels to /boot > partition from backup media (formerly in /boot of / partition) > (6) make sure /boot partition is mounted in fstab > (7) /boot partition should also be in the RAID1 > > Does that sound like it would fix it? Any ideas? > > More info below, if you're interested. ;-) From tclug1 at whitleymott.net Thu Sep 3 09:43:20 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Thu, 3 Sep 2015 09:43:20 -0500 Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") In-Reply-To: References: <03d18db2b12b27809a0daa0bcbad7b23.squirrel@www.ron-l-j.com> Message-ID: you want to upgrade. seems to me the easiest, quickest, and most trustworthy way to proceed is reinstall. experience taught me long ago reinstalling is your friend, far easier than sorting out a confusing mess. also, many major upgrades haven't worked properly without reinstalling. sometimes raid formats have changed, sometimes the upgrade process just leaves a nonworking mess and you need to reinstall anyway. and while you're at it setup your discs so you have 2 complete and distinct installations simultaneously. that way the next time around you can install the next before/without overwriting what's working. it's easiest to use the installer to do your partitioning. i have 2 discs and build raid1 sets by setting the 2 discs up with identical partitions, on each disc i put two 300mb boot partitions, two 10gig swap partitions, and the rest of the space equally divided between two lvm partitions. the swap could be within the lvm, but there have been times i've been happy to have partitions i could play with without destroying either installation. if you want more swap later you can add additional swap from within the lvm. you can even change your mind later about having 2 sets and absorb the second lvm into the first. just like you can add more salt later but you can't take out what's already in the stew. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhubarbpieguy at gmail.com Thu Sep 3 11:05:54 2015 From: rhubarbpieguy at gmail.com (rhubarbpieguy at gmail.com) Date: Thu, 3 Sep 2015 11:05:54 -0500 Subject: [tclug-list] Dual booting with Windows 8.1. What do the default partitions mean? Message-ID: <55E86FE2.2080307@gmail.com> I recently purchased a new box on which I wiped the drive and installed Linux using efi. My efi partition is sda1, with Linux distributions on sda[2-3] and sda4 as swap. However, if I don't have Windows, is there an advantage of using efi? If I understand, I can enable legacy BIOS and lose the efi partition. In that case I could have Linux distributions on sda[1-3]. I believe efi has effectively no partition limitations, so I could have efi on sda1, Linux distributions on sda[2-3] and sda5 as swap. But I don't see the advantage of efi as four partitions is enough for me. Also, while I'll probably keep my current setup, I realize the danger of not trying new things. I'd like to attempt dual booting with Windows but am confused. The default partitions were: * free space 1 MB * /dev/sda1 ntfs 1073 MB * /dev/sda2 efi 104 MB * /dev/sda3 ntfs 134 MB * /dev/sda4 ntfs 489243 MB * /dev/sda5 ntfs 9550 MB * free space 0 MB I understand sda2 and assume sda4 is the main Windows partition. But what were sda3 and sda5? Can I delete them? Can I resize sda4 during my Linux installation? I've not booted to Windows since 2002 and probably still won't after this experiment. But I'd be interested in Linux users' input. From mbmiller+l at gmail.com Thu Sep 3 14:55:00 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Thu, 3 Sep 2015 14:55:00 -0500 (CDT) Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") In-Reply-To: References: <03d18db2b12b27809a0daa0bcbad7b23.squirrel@www.ron-l-j.com> Message-ID: On Thu, 3 Sep 2015, gregrwm wrote: > you want to upgrade. seems to me the easiest, quickest, and most > trustworthy way to proceed is reinstall. That makes sense. I have to make sure I get the instructions right this time. I seem to have missed a step last time. > and while you're at it setup your discs so you have 2 complete and > distinct installations simultaneously. that way the next time around > you can install the next before/without overwriting what's working. I have no idea how to do that. > i have 2 discs and build raid1 sets by setting the 2 discs up with > identical partitions, on each disc i put two 300mb boot partitions, two > 10gig swap partitions, and the rest of the space equally divided between > two lvm partitions. I wouldn't know how to do that, either. Suppose I have two 3 TB disks, sda and sdb and I just want RAID1 with 300 MB /boot, 32 GB swap and the rest in /. How do I do it? I posted very detailed instructions explaining exactly how I did it several years ago, but now I know that those instructions were flawed. Can you figure out what I did wrong here?: http://shadowknight.real-time.com/pipermail/tclug-list/2012-May/062090.html This part seems especially suspicious: "Note that I did not add /boot to the RAID -- that will cause Ubuntu to fail to load GRUB, so don't do that." Maybe /boot has to be added afterward? Or maybe that's just wrong. I'm sure I read it somewhere. What seems to have happened is that Ubuntu made a /boot directory in the / partition of the RAID, but that did not work. Mike From mbmiller+l at gmail.com Thu Sep 3 15:17:35 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Thu, 3 Sep 2015 15:17:35 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups Message-ID: How to format? I have a couple of Linux boxes that I would like to regularly backup to a 5 TB external drive. It seems like it would be a good idea to format that drive with ext4. Can I just do that with gparted? The drive comes with NTFS format. Are there any issues I should know about? Which directories to back up? What really needs to be backed up? I guess if the system totally failed I'd install Linux (Ubuntu) again. Of course /home is needed, but /usr/local and /opt often have programs I've installed and /etc will have a bunch of settings. I guess /var can have some important stuff. Are crontabs stored in /var? Which software to use for backup? I guess I want only to have in backup what is on the originating drive. So if I have deleted a file, I want it to be deleted on the backup drive, too. I assume rsync can do this. Would this be correct?: rsync -av --update --delete /home /usr/local /etc /var /opt /media/me/back TIA! Mike From tlunde at gmail.com Fri Sep 4 08:21:52 2015 From: tlunde at gmail.com (T L) Date: Fri, 4 Sep 2015 08:21:52 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: Assuming that you have NOTHING on the drive that you care about, I would remove the factory partitioning and create a new GPT table with parted. Then, format that as ext4. On Sep 3, 2015 3:17 PM, "Mike Miller" wrote: > How to format? > > I have a couple of Linux boxes that I would like to regularly backup to a > 5 TB external drive. It seems like it would be a good idea to format that > drive with ext4. Can I just do that with gparted? The drive comes with > NTFS format. Are there any issues I should know about? > > > Which directories to back up? > > What really needs to be backed up? I guess if the system totally failed > I'd install Linux (Ubuntu) again. Of course /home is needed, but > /usr/local and /opt often have programs I've installed and /etc will have a > bunch of settings. I guess /var can have some important stuff. Are > crontabs stored in /var? > > > Which software to use for backup? > > I guess I want only to have in backup what is on the originating drive. So > if I have deleted a file, I want it to be deleted on the backup drive, > too. I assume rsync can do this. Would this be correct?: > > rsync -av --update --delete /home /usr/local /etc /var /opt /media/me/back > > > TIA! > > Mike > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapinjeff at gmail.com Fri Sep 4 13:08:41 2015 From: chapinjeff at gmail.com (Jeff Chapin) Date: Fri, 4 Sep 2015 13:08:41 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: Looking at the rsync command you gave, it looks correct -- but rsync can do so much more when backing up! Using the magic of rsync, and the magic of hardlinks, you can make a full backup, in incremental time and space. Rsync has, built into it, the ability to compare your most recent backup files with existing backup files, and if they are they same, use a hard link, and copy them over if they differ. This allows you to store just the files that change -- but it looks like a full backup every time it runs. This way, you can keep, say, hourly backups for the last week -- and recover an accidentally deleted or altered file, even after the latest backup has run. For more details: https://blog.interlinked.org/tutorials/rsync_time_machine.html On Fri, Sep 4, 2015 at 8:21 AM, T L wrote: > Assuming that you have NOTHING on the drive that you care about, I would > remove the factory partitioning and create a new GPT table with parted. > > Then, format that as ext4. > On Sep 3, 2015 3:17 PM, "Mike Miller" wrote: > >> How to format? >> >> I have a couple of Linux boxes that I would like to regularly backup to a >> 5 TB external drive. It seems like it would be a good idea to format that >> drive with ext4. Can I just do that with gparted? The drive comes with >> NTFS format. Are there any issues I should know about? >> >> >> Which directories to back up? >> >> What really needs to be backed up? I guess if the system totally failed >> I'd install Linux (Ubuntu) again. Of course /home is needed, but >> /usr/local and /opt often have programs I've installed and /etc will have a >> bunch of settings. I guess /var can have some important stuff. Are >> crontabs stored in /var? >> >> >> Which software to use for backup? >> >> I guess I want only to have in backup what is on the originating drive. >> So if I have deleted a file, I want it to be deleted on the backup drive, >> too. I assume rsync can do this. Would this be correct?: >> >> rsync -av --update --delete /home /usr/local /etc /var /opt /media/me/back >> >> >> TIA! >> >> Mike >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > -- Jeff Chapin President, CedarLug, retired President, UNIPC, "I'll get around to it" President, UNI Scuba Club Senator, NISG, retired -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbmiller+l at gmail.com Fri Sep 4 14:09:22 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Fri, 4 Sep 2015 14:09:22 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: On Fri, 4 Sep 2015, T L wrote: > Assuming that you have NOTHING on the drive that you care about, I would > remove the factory partitioning and create a new GPT table with parted. Right. It's just a new drive - Seagate Backup+ 5 TB (Costco $140).[1] I think parted is pretty easy to use, once you know what to do. I guess I need to use parted -l to find the /dev/sd letter (I'll use /dev/sdx here), then something like this will create the ext4 partition: sudo parted /dev/sdx mklabel gpt sudo parted /dev/sdx mkpart primary ext4 0 5TiB I think it then needs to be formatted. Would this command do it? mkfs.ext4 /dev/sdx1 Mike [1] Those prices keep coming down. I can still remember my first hard drive -- a 20 MB MFA drive formatted to 30 MB with an RLL controller. Only $400. So in 1987 the price of storage was 714,285 times per byte what I'm paying today. Mike From mbmiller+l at gmail.com Fri Sep 4 14:17:11 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Fri, 4 Sep 2015 14:17:11 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: That is really cool! I'll have to try something like that. I'm thinking a good strategy is to have two drives, both with all the same stuff on them, and I'll use them both to backup all my Linux boxes (home, office, laptops). I'll just switch between home and office every week or so. That way if my house burns down or my office is burglarized, I still have a copy of everything from last week at the other location. Does that seem reasonable? The thing I'm not sure of is how that strategy would work with the "time machine" concept -- I'd be using two drives and swapping them weekly. Mike On Fri, 4 Sep 2015, Jeff Chapin wrote: > Looking at the rsync command you gave, it looks correct -- but rsync can do > so much more when backing up! > > Using the magic of rsync, and the magic of hardlinks, you can make a full > backup, in incremental time and space. Rsync has, built into it, the > ability to compare your most recent backup files with existing backup > files, and if they are they same, use a hard link, and copy them over if > they differ. This allows you to store just the files that change -- but it > looks like a full backup every time it runs. This way, you can keep, say, > hourly backups for the last week -- and recover an accidentally deleted or > altered file, even after the latest backup has run. > > For more details: > https://blog.interlinked.org/tutorials/rsync_time_machine.html > > > > On Fri, Sep 4, 2015 at 8:21 AM, T L wrote: > >> Assuming that you have NOTHING on the drive that you care about, I would >> remove the factory partitioning and create a new GPT table with parted. >> >> Then, format that as ext4. >> On Sep 3, 2015 3:17 PM, "Mike Miller" wrote: >> >>> How to format? >>> >>> I have a couple of Linux boxes that I would like to regularly backup to a >>> 5 TB external drive. It seems like it would be a good idea to format that >>> drive with ext4. Can I just do that with gparted? The drive comes with >>> NTFS format. Are there any issues I should know about? >>> >>> >>> Which directories to back up? >>> >>> What really needs to be backed up? I guess if the system totally failed >>> I'd install Linux (Ubuntu) again. Of course /home is needed, but >>> /usr/local and /opt often have programs I've installed and /etc will have a >>> bunch of settings. I guess /var can have some important stuff. Are >>> crontabs stored in /var? >>> >>> >>> Which software to use for backup? >>> >>> I guess I want only to have in backup what is on the originating drive. >>> So if I have deleted a file, I want it to be deleted on the backup drive, >>> too. I assume rsync can do this. Would this be correct?: >>> >>> rsync -av --update --delete /home /usr/local /etc /var /opt /media/me/back >>> >>> >>> TIA! >>> >>> Mike >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>> >> >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> >> > > > -- > Jeff Chapin > President, CedarLug, retired > President, UNIPC, "I'll get around to it" > President, UNI Scuba Club > Senator, NISG, retired > From tclug1 at whitleymott.net Fri Sep 4 14:24:39 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Fri, 4 Sep 2015 14:24:39 -0500 Subject: [tclug-list] fixing bad /boot in RAID1 (was "bad magic number") Message-ID: > > and while you're at it setup your discs so you have 2 complete and >> distinct installations simultaneously. that way the next time around you >> can install the next before/without overwriting what's working. >> > > I have no idea how to do that. > > i have 2 discs and build raid1 sets by setting the 2 discs up with >> identical partitions, on each disc i put two 300mb boot partitions, two >> 10gig swap partitions, and the rest of the space equally divided between >> two lvm partitions. >> > > I wouldn't know how to do that, either. > > Suppose I have two 3 TB disks, sda and sdb and I just want RAID1 with 300 > MB /boot, 32 GB swap and the rest in /. How do I do it? > in the installer choose "manual" partitioning, delete existing partitions, on sda create a 300mb partition (for /boot), another 300mb partition (for /boot for a second installation), a 32gb partition (or however much swap you want), another 32gb partition (for swap a second installation), a partition given half all remaining space (for lvm), another partition given the remaining space (for a second lvm). then do all the same on sdb. i don't remember which installer has a feature to do that in one stroke. (someday perhaps an installer will be enhanced to make use of partitionable raid, if available presumably that would make this process even easier.) then create your raid1 sets, matching one partition from sda and one from sdb for each set, a 300mb raidset, specify it to be for /boot, a 32gb raidset, specify it to be for swap, an lvm raidset, within which you create a logical volume of whatever size you want for / no need to bother to make raidsets of the second set of partitions yet, just be happy they're the salt you didn't put into the stew. the installer should just do the right thing by itself re GPT, partition types, and aligning and spacing of partitions. -------------- next part -------------- An HTML attachment was scrubbed... URL: From chapinjeff at gmail.com Fri Sep 4 14:28:16 2015 From: chapinjeff at gmail.com (Jeff Chapin) Date: Fri, 4 Sep 2015 14:28:16 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: The script would take a little tweaking, but it could work. I, personally, know that I would get lazy at some point and fail to swap the drives for months on end. I would consider taking one of the drives to both locations, and getting an initial backup of each location, and then mirroring that to the other drive -- and then have both locations back up to both drives. Alternatively, you could backup both drives to the local drive, and then mirror the two drives(you could do hourly local backups, and nightly remote copies). Since rsync only transfers the differences, once you have the initial backup, it's likely that each day's change is fairly small. If you use the flag to make rsync aware of the hardlinks, you could presumably replicate a full copy of the day's hourly backups fairly quickly. On Fri, Sep 4, 2015 at 2:17 PM, Mike Miller wrote: > That is really cool! I'll have to try something like that. I'm thinking a > good strategy is to have two drives, both with all the same stuff on them, > and I'll use them both to backup all my Linux boxes (home, office, > laptops). I'll just switch between home and office every week or so. That > way if my house burns down or my office is burglarized, I still have a copy > of everything from last week at the other location. > > Does that seem reasonable? The thing I'm not sure of is how that strategy > would work with the "time machine" concept -- I'd be using two drives and > swapping them weekly. > > Mike > > > > On Fri, 4 Sep 2015, Jeff Chapin wrote: > > Looking at the rsync command you gave, it looks correct -- but rsync can do >> so much more when backing up! >> >> Using the magic of rsync, and the magic of hardlinks, you can make a full >> backup, in incremental time and space. Rsync has, built into it, the >> ability to compare your most recent backup files with existing backup >> files, and if they are they same, use a hard link, and copy them over if >> they differ. This allows you to store just the files that change -- but it >> looks like a full backup every time it runs. This way, you can keep, say, >> hourly backups for the last week -- and recover an accidentally deleted or >> altered file, even after the latest backup has run. >> >> For more details: >> https://blog.interlinked.org/tutorials/rsync_time_machine.html >> >> >> >> On Fri, Sep 4, 2015 at 8:21 AM, T L wrote: >> >> Assuming that you have NOTHING on the drive that you care about, I would >>> remove the factory partitioning and create a new GPT table with parted. >>> >>> Then, format that as ext4. >>> On Sep 3, 2015 3:17 PM, "Mike Miller" wrote: >>> >>> How to format? >>>> >>>> I have a couple of Linux boxes that I would like to regularly backup to >>>> a >>>> 5 TB external drive. It seems like it would be a good idea to format >>>> that >>>> drive with ext4. Can I just do that with gparted? The drive comes with >>>> NTFS format. Are there any issues I should know about? >>>> >>>> >>>> Which directories to back up? >>>> >>>> What really needs to be backed up? I guess if the system totally failed >>>> I'd install Linux (Ubuntu) again. Of course /home is needed, but >>>> /usr/local and /opt often have programs I've installed and /etc will >>>> have a >>>> bunch of settings. I guess /var can have some important stuff. Are >>>> crontabs stored in /var? >>>> >>>> >>>> Which software to use for backup? >>>> >>>> I guess I want only to have in backup what is on the originating drive. >>>> So if I have deleted a file, I want it to be deleted on the backup >>>> drive, >>>> too. I assume rsync can do this. Would this be correct?: >>>> >>>> rsync -av --update --delete /home /usr/local /etc /var /opt >>>> /media/me/back >>>> >>>> >>>> TIA! >>>> >>>> Mike >>>> _______________________________________________ >>>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>>> tclug-list at mn-linux.org >>>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>>> >>>> >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>> >>> >>> >> >> -- >> Jeff Chapin >> President, CedarLug, retired >> President, UNIPC, "I'll get around to it" >> President, UNI Scuba Club >> Senator, NISG, retired >> >> _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > -- Jeff Chapin President, CedarLug, retired President, UNIPC, "I'll get around to it" President, UNI Scuba Club Senator, NISG, retired -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbmiller+l at gmail.com Fri Sep 4 14:53:10 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Fri, 4 Sep 2015 14:53:10 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: On Fri, 4 Sep 2015, Jeff Chapin wrote: > The script would take a little tweaking, but it could work. > > I, personally, know that I would get lazy at some point and fail to swap > the drives for months on end. That could happen, but at least the drive is backed up to its attached external drive. > I would consider taking one of the drives to both locations, and getting > an initial backup of each location, and then mirroring that to the other > drive That part sounds OK, but... > -- and then have both locations back up to both drives. Do you mean that I should do backups over the internet? If so, I don't think I want to do that because of the bandwidth. > Alternatively, you could backup both drives to the local drive, and then > mirror the two drives (you could do hourly local backups, and nightly > remote copies). Since rsync only transfers the differences, once you > have the initial backup, it's likely that each day's change is fairly > small. If you use the flag to make rsync aware of the hardlinks, you > could presumably replicate a full copy of the day's hourly backups > fairly quickly. It is starting to get complicated! I love the idea of doing really cool, cutting-edge things, but I've been learning that it can take too much time away from other stuff I want to do. Maybe I can start out like you said -- back up to one, mirror to the other -- then just skip the time machine and do ordinary updates to the attached drive... rsync -av --update --delete /home /usr/local /etc /var /opt /media/me/back ...but also at least see if backing up also to the remote location is feasible: rsync -av --update --delete /home /usr/local /etc /var /opt remote:/media/me/back The script would have to have a way of checking for the existence of the attached hard drive both locally and remotely. Would it be a lot more work to add in the "time machine" functionality? Mike > On Fri, Sep 4, 2015 at 2:17 PM, Mike Miller wrote: > >> That is really cool! I'll have to try something like that. I'm thinking a >> good strategy is to have two drives, both with all the same stuff on them, >> and I'll use them both to backup all my Linux boxes (home, office, >> laptops). I'll just switch between home and office every week or so. That >> way if my house burns down or my office is burglarized, I still have a copy >> of everything from last week at the other location. >> >> Does that seem reasonable? The thing I'm not sure of is how that strategy >> would work with the "time machine" concept -- I'd be using two drives and >> swapping them weekly. >> >> Mike >> >> >> >> On Fri, 4 Sep 2015, Jeff Chapin wrote: >> >> Looking at the rsync command you gave, it looks correct -- but rsync can do >>> so much more when backing up! >>> >>> Using the magic of rsync, and the magic of hardlinks, you can make a full >>> backup, in incremental time and space. Rsync has, built into it, the >>> ability to compare your most recent backup files with existing backup >>> files, and if they are they same, use a hard link, and copy them over if >>> they differ. This allows you to store just the files that change -- but it >>> looks like a full backup every time it runs. This way, you can keep, say, >>> hourly backups for the last week -- and recover an accidentally deleted or >>> altered file, even after the latest backup has run. >>> >>> For more details: >>> https://blog.interlinked.org/tutorials/rsync_time_machine.html >>> >>> >>> >>> On Fri, Sep 4, 2015 at 8:21 AM, T L wrote: >>> >>> Assuming that you have NOTHING on the drive that you care about, I would >>>> remove the factory partitioning and create a new GPT table with parted. >>>> >>>> Then, format that as ext4. >>>> On Sep 3, 2015 3:17 PM, "Mike Miller" wrote: >>>> >>>> How to format? >>>>> >>>>> I have a couple of Linux boxes that I would like to regularly backup to >>>>> a >>>>> 5 TB external drive. It seems like it would be a good idea to format >>>>> that >>>>> drive with ext4. Can I just do that with gparted? The drive comes with >>>>> NTFS format. Are there any issues I should know about? >>>>> >>>>> >>>>> Which directories to back up? >>>>> >>>>> What really needs to be backed up? I guess if the system totally failed >>>>> I'd install Linux (Ubuntu) again. Of course /home is needed, but >>>>> /usr/local and /opt often have programs I've installed and /etc will >>>>> have a >>>>> bunch of settings. I guess /var can have some important stuff. Are >>>>> crontabs stored in /var? >>>>> >>>>> >>>>> Which software to use for backup? >>>>> >>>>> I guess I want only to have in backup what is on the originating drive. >>>>> So if I have deleted a file, I want it to be deleted on the backup >>>>> drive, >>>>> too. I assume rsync can do this. Would this be correct?: >>>>> >>>>> rsync -av --update --delete /home /usr/local /etc /var /opt >>>>> /media/me/back >>>>> >>>>> >>>>> TIA! >>>>> >>>>> Mike >>>>> _______________________________________________ >>>>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>>>> tclug-list at mn-linux.org >>>>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>>>> >>>>> >>>> _______________________________________________ >>>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>>> tclug-list at mn-linux.org >>>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>>> >>>> >>>> >>> >>> -- >>> Jeff Chapin >>> President, CedarLug, retired >>> President, UNIPC, "I'll get around to it" >>> President, UNI Scuba Club >>> Senator, NISG, retired >>> >>> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> > > > > -- > Jeff Chapin > President, CedarLug, retired > President, UNIPC, "I'll get around to it" > President, UNI Scuba Club > Senator, NISG, retired > From lkateley at kateley.com Fri Sep 4 15:44:14 2015 From: lkateley at kateley.com (Linda Kateley) Date: Fri, 4 Sep 2015 15:44:14 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: Message-ID: <55EA029E.6020707@kateley.com> So.. I have to weigh in here.. I don't know if people know i do the classes for freenas, but you can use it as a gui for both formatting drives and replication.. and it's free. and it has time machine.. In my environment I just has a small nas box that i backup to all of my systems including my macs. I just saw rsync.net will take zfs receives and charge .06 per GB per month for zfs. linda On 9/4/15 2:28 PM, Jeff Chapin wrote: > The script would take a little tweaking, but it could work. > > I, personally, know that I would get lazy at some point and fail to > swap the drives for months on end. I would consider taking one of the > drives to both locations, and getting an initial backup of each > location, and then mirroring that to the other drive -- and then have > both locations back up to both drives. Alternatively, you could backup > both drives to the local drive, and then mirror the two drives(you > could do hourly local backups, and nightly remote copies). Since rsync > only transfers the differences, once you have the initial backup, it's > likely that each day's change is fairly small. If you use the flag to > make rsync aware of the hardlinks, you could presumably replicate a > full copy of the day's hourly backups fairly quickly. > > > On Fri, Sep 4, 2015 at 2:17 PM, Mike Miller > wrote: > > That is really cool! I'll have to try something like that. I'm > thinking a good strategy is to have two drives, both with all the > same stuff on them, and I'll use them both to backup all my Linux > boxes (home, office, laptops). I'll just switch between home and > office every week or so. That way if my house burns down or my > office is burglarized, I still have a copy of everything from last > week at the other location. > > Does that seem reasonable? The thing I'm not sure of is how that > strategy would work with the "time machine" concept -- I'd be > using two drives and swapping them weekly. > > Mike > > > > On Fri, 4 Sep 2015, Jeff Chapin wrote: > > Looking at the rsync command you gave, it looks correct -- but > rsync can do > so much more when backing up! > > Using the magic of rsync, and the magic of hardlinks, you can > make a full > backup, in incremental time and space. Rsync has, built into > it, the > ability to compare your most recent backup files with existing > backup > files, and if they are they same, use a hard link, and copy > them over if > they differ. This allows you to store just the files that > change -- but it > looks like a full backup every time it runs. This way, you can > keep, say, > hourly backups for the last week -- and recover an > accidentally deleted or > altered file, even after the latest backup has run. > > For more details: > https://blog.interlinked.org/tutorials/rsync_time_machine.html > > > > On Fri, Sep 4, 2015 at 8:21 AM, T L > wrote: > > Assuming that you have NOTHING on the drive that you care > about, I would > remove the factory partitioning and create a new GPT table > with parted. > > Then, format that as ext4. > On Sep 3, 2015 3:17 PM, "Mike Miller" > > wrote: > > How to format? > > I have a couple of Linux boxes that I would like to > regularly backup to a > 5 TB external drive. It seems like it would be a good > idea to format that > drive with ext4. Can I just do that with gparted? > The drive comes with > NTFS format. Are there any issues I should know about? > > > Which directories to back up? > > What really needs to be backed up? I guess if the > system totally failed > I'd install Linux (Ubuntu) again. Of course /home is > needed, but > /usr/local and /opt often have programs I've installed > and /etc will have a > bunch of settings. I guess /var can have some > important stuff. Are > crontabs stored in /var? > > > Which software to use for backup? > > I guess I want only to have in backup what is on the > originating drive. > So if I have deleted a file, I want it to be deleted > on the backup drive, > too. I assume rsync can do this. Would this be correct?: > > rsync -av --update --delete /home /usr/local /etc /var > /opt /media/me/back > > > TIA! > > Mike > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > > > > -- > Jeff Chapin > President, CedarLug, retired > President, UNIPC, "I'll get around to it" > President, UNI Scuba Club > Senator, NISG, retired > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > > > > -- > Jeff Chapin > President, CedarLug, retired > President, UNIPC, "I'll get around to it" > President, UNI Scuba Club > Senator, NISG, retired > > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list -- Linda Kateley Kateley Company Skype ID-kateleyco http://kateleyco.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbmiller+l at gmail.com Fri Sep 4 16:40:46 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Fri, 4 Sep 2015 16:40:46 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: <55EA029E.6020707@kateley.com> References: <55EA029E.6020707@kateley.com> Message-ID: I have the impression that zfs is a safer and more reliable system than ext4, so maybe I should want zfs, but is it working on Linux? Is it hard to get it installed for use with an external drive? I guess $0.06/GB-month is a good deal if you have a small amount of data to backup. Mike On Fri, 4 Sep 2015, Linda Kateley wrote: > So.. I have to weigh in here.. I don't know if people know i do the classes > for freenas, but you can use it as a gui for both formatting drives and > replication.. and it's free. and it has time machine.. > > In my environment I just has a small nas box that i backup to all of my > systems including my macs. > > I just saw rsync.net will take zfs receives and charge .06 per GB per month > for zfs. > > linda > > On 9/4/15 2:28 PM, Jeff Chapin wrote: >> The script would take a little tweaking, but it could work. >> >> I, personally, know that I would get lazy at some point and fail to swap >> the drives for months on end. I would consider taking one of the drives to >> both locations, and getting an initial backup of each location, and then >> mirroring that to the other drive -- and then have both locations back up >> to both drives. Alternatively, you could backup both drives to the local >> drive, and then mirror the two drives(you could do hourly local backups, >> and nightly remote copies). Since rsync only transfers the differences, >> once you have the initial backup, it's likely that each day's change is >> fairly small. If you use the flag to make rsync aware of the hardlinks, you >> could presumably replicate a full copy of the day's hourly backups fairly >> quickly. >> >> >> On Fri, Sep 4, 2015 at 2:17 PM, Mike Miller > > wrote: >> >> That is really cool! I'll have to try something like that. I'm >> thinking a good strategy is to have two drives, both with all the >> same stuff on them, and I'll use them both to backup all my Linux >> boxes (home, office, laptops). I'll just switch between home and >> office every week or so. That way if my house burns down or my >> office is burglarized, I still have a copy of everything from last >> week at the other location. >> >> Does that seem reasonable? The thing I'm not sure of is how that >> strategy would work with the "time machine" concept -- I'd be >> using two drives and swapping them weekly. >> >> Mike >> >> >> >> On Fri, 4 Sep 2015, Jeff Chapin wrote: >> >> Looking at the rsync command you gave, it looks correct -- but >> rsync can do >> so much more when backing up! >> >> Using the magic of rsync, and the magic of hardlinks, you can >> make a full >> backup, in incremental time and space. Rsync has, built into >> it, the >> ability to compare your most recent backup files with existing >> backup >> files, and if they are they same, use a hard link, and copy >> them over if >> they differ. This allows you to store just the files that >> change -- but it >> looks like a full backup every time it runs. This way, you can >> keep, say, >> hourly backups for the last week -- and recover an >> accidentally deleted or >> altered file, even after the latest backup has run. >> >> For more details: >> https://blog.interlinked.org/tutorials/rsync_time_machine.html >> >> >> >> On Fri, Sep 4, 2015 at 8:21 AM, T L > > wrote: >> >> Assuming that you have NOTHING on the drive that you care >> about, I would >> remove the factory partitioning and create a new GPT table >> with parted. >> >> Then, format that as ext4. >> On Sep 3, 2015 3:17 PM, "Mike Miller" >> > wrote: >> >> How to format? >> >> I have a couple of Linux boxes that I would like to >> regularly backup to a >> 5 TB external drive. It seems like it would be a good >> idea to format that >> drive with ext4. Can I just do that with gparted? >> The drive comes with >> NTFS format. Are there any issues I should know about? >> >> >> Which directories to back up? >> >> What really needs to be backed up? I guess if the >> system totally failed >> I'd install Linux (Ubuntu) again. Of course /home is >> needed, but >> /usr/local and /opt often have programs I've installed >> and /etc will have a >> bunch of settings. I guess /var can have some >> important stuff. Are >> crontabs stored in /var? >> >> >> Which software to use for backup? >> >> I guess I want only to have in backup what is on the >> originating drive. >> So if I have deleted a file, I want it to be deleted >> on the backup drive, >> too. I assume rsync can do this. Would this be correct?: >> >> rsync -av --update --delete /home /usr/local /etc /var >> /opt /media/me/back >> >> >> TIA! >> >> Mike >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> >> >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> >> >> >> >> -- Jeff Chapin >> President, CedarLug, retired >> President, UNIPC, "I'll get around to it" >> President, UNI Scuba Club >> Senator, NISG, retired >> >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> >> >> >> >> -- >> Jeff Chapin >> President, CedarLug, retired >> President, UNIPC, "I'll get around to it" >> President, UNI Scuba Club >> Senator, NISG, retired >> >> >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > -- > Linda Kateley > Kateley Company > Skype ID-kateleyco > http://kateleyco.com > > From tclug at freakzilla.com Fri Sep 4 16:46:08 2015 From: tclug at freakzilla.com (Clug) Date: Fri, 4 Sep 2015 16:46:08 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: <55EA029E.6020707@kateley.com> Message-ID: ZFS is easy and awesome but VERY resource-heavy. You want 1GB of RAM for each TB in your array, and you want that to be ECC RAM! Which means a motherboard/CPU that supports that. I'm running without ECC RAM. No real issues, really, but my system is INSANELY slow when doing scrubs/repairs (and yes, I fixed those parameters). On Fri, 4 Sep 2015, Mike Miller wrote: > I have the impression that zfs is a safer and more reliable system than ext4, > so maybe I should want zfs, but is it working on Linux? Is it hard to get it > installed for use with an external drive? > > I guess $0.06/GB-month is a good deal if you have a small amount of data to > backup. > > Mike > > > On Fri, 4 Sep 2015, Linda Kateley wrote: > >> So.. I have to weigh in here.. I don't know if people know i do the classes >> for freenas, but you can use it as a gui for both formatting drives and >> replication.. and it's free. and it has time machine.. >> >> In my environment I just has a small nas box that i backup to all of my >> systems including my macs. >> >> I just saw rsync.net will take zfs receives and charge .06 per GB per month >> for zfs. >> >> linda >> >> On 9/4/15 2:28 PM, Jeff Chapin wrote: >>> The script would take a little tweaking, but it could work. >>> >>> I, personally, know that I would get lazy at some point and fail to swap >>> the drives for months on end. I would consider taking one of the drives to >>> both locations, and getting an initial backup of each location, and then >>> mirroring that to the other drive -- and then have both locations back up >>> to both drives. Alternatively, you could backup both drives to the local >>> drive, and then mirror the two drives(you could do hourly local backups, >>> and nightly remote copies). Since rsync only transfers the differences, >>> once you have the initial backup, it's likely that each day's change is >>> fairly small. If you use the flag to make rsync aware of the hardlinks, >>> you could presumably replicate a full copy of the day's hourly backups >>> fairly quickly. >>> >>> >>> On Fri, Sep 4, 2015 at 2:17 PM, Mike Miller >> > wrote: >>> >>> That is really cool! I'll have to try something like that. I'm >>> thinking a good strategy is to have two drives, both with all the >>> same stuff on them, and I'll use them both to backup all my Linux >>> boxes (home, office, laptops). I'll just switch between home and >>> office every week or so. That way if my house burns down or my >>> office is burglarized, I still have a copy of everything from last >>> week at the other location. >>> >>> Does that seem reasonable? The thing I'm not sure of is how that >>> strategy would work with the "time machine" concept -- I'd be >>> using two drives and swapping them weekly. >>> >>> Mike >>> >>> >>> >>> On Fri, 4 Sep 2015, Jeff Chapin wrote: >>> >>> Looking at the rsync command you gave, it looks correct -- but >>> rsync can do >>> so much more when backing up! >>> >>> Using the magic of rsync, and the magic of hardlinks, you can >>> make a full >>> backup, in incremental time and space. Rsync has, built into >>> it, the >>> ability to compare your most recent backup files with existing >>> backup >>> files, and if they are they same, use a hard link, and copy >>> them over if >>> they differ. This allows you to store just the files that >>> change -- but it >>> looks like a full backup every time it runs. This way, you can >>> keep, say, >>> hourly backups for the last week -- and recover an >>> accidentally deleted or >>> altered file, even after the latest backup has run. >>> >>> For more details: >>> https://blog.interlinked.org/tutorials/rsync_time_machine.html >>> >>> >>> >>> On Fri, Sep 4, 2015 at 8:21 AM, T L >> > wrote: >>> >>> Assuming that you have NOTHING on the drive that you care >>> about, I would >>> remove the factory partitioning and create a new GPT table >>> with parted. >>> >>> Then, format that as ext4. >>> On Sep 3, 2015 3:17 PM, "Mike Miller" >>> > wrote: >>> >>> How to format? >>> >>> I have a couple of Linux boxes that I would like to >>> regularly backup to a >>> 5 TB external drive. It seems like it would be a good >>> idea to format that >>> drive with ext4. Can I just do that with gparted? The >>> drive comes with >>> NTFS format. Are there any issues I should know about? >>> >>> >>> Which directories to back up? >>> >>> What really needs to be backed up? I guess if the >>> system totally failed >>> I'd install Linux (Ubuntu) again. Of course /home is >>> needed, but >>> /usr/local and /opt often have programs I've installed >>> and /etc will have a >>> bunch of settings. I guess /var can have some >>> important stuff. Are >>> crontabs stored in /var? >>> >>> >>> Which software to use for backup? >>> >>> I guess I want only to have in backup what is on the >>> originating drive. >>> So if I have deleted a file, I want it to be deleted >>> on the backup drive, >>> too. I assume rsync can do this. Would this be correct?: >>> >>> rsync -av --update --delete /home /usr/local /etc /var >>> /opt /media/me/back >>> >>> >>> TIA! >>> >>> Mike >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>> >>> >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>> >>> >>> >>> >>> -- Jeff Chapin >>> President, CedarLug, retired >>> President, UNIPC, "I'll get around to it" >>> President, UNI Scuba Club >>> Senator, NISG, retired >>> >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >>> >>> >>> >>> >>> -- >>> Jeff Chapin >>> President, CedarLug, retired >>> President, UNIPC, "I'll get around to it" >>> President, UNI Scuba Club >>> Senator, NISG, retired >>> >>> >>> _______________________________________________ >>> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >>> tclug-list at mn-linux.org >>> http://mailman.mn-linux.org/mailman/listinfo/tclug-list >> >> -- >> Linda Kateley >> Kateley Company >> Skype ID-kateleyco >> http://kateleyco.com >> >> > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > From tclug1 at whitleymott.net Fri Sep 4 17:31:05 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Fri, 4 Sep 2015 17:31:05 -0500 Subject: [tclug-list] seeking satisfying sip Message-ID: seeking a satisfactory sip on android. my d710 has jellybean 4.1.2 gb27 freedompop, my t989 has kitkat 4.4.4 cm11-20150823 wifi only. zoiper works on both, but zoiper (beta 1.24.9) supports neither bluetooth nor tcp. csipsimple 1.02.03 often works on my d710, but the other party consistently gets severe echo of their own voice (echo cancellation is checked), and it often doesn't re-register when going back and forth between wifi and tower reception. it won't register at all on my t989/kitkat. sipdroid 3.7beta is slightly better about re-registering on my d710, but still not as consistent as i'd expect, uses significantly more battery than csipsimple, and also fails to register at all on my t989/kitkat. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tclug1 at whitleymott.net Fri Sep 4 19:41:02 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Fri, 4 Sep 2015 19:41:02 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: <55EA029E.6020707@kateley.com> Message-ID: BackupPC has been doing my own and my clients' backups for years now, i'm not aware of anything that competes. it keeps a hashed hardlinked pool, optionally also compressed (you can choose any compressor), such that there is only one copy of any file that stays the same across backups, or appears multiple places in your hierarchies. it will pull from other linux' or windoz' or macs via rsync or tar or samba. backups can be aged out and automatically weeded out according to a flexible binary reduction formula so you can keep very old and very new backups and thin down the ones in between. all free&libre. once you took your first full backups, your incrementals could be so lightweight you might be happy to do them over the internet. or of course you could swap the drives back and forth manually. initial setup time, ie reading into it, is an investment to be sure, tho i would wager it to be significantly less than writing and debugging your own script, which would also no doubt be far less capable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From woodbrian77 at gmail.com Mon Sep 7 14:49:36 2015 From: woodbrian77 at gmail.com (Brian Wood) Date: Mon, 7 Sep 2015 14:49:36 -0500 Subject: [tclug-list] OT: Acer Travelmate for sale Message-ID: Shalom I've reduced the price of the Acer laptop that I mentioned here a few months ago to $165 or best offer. Weighs 4.4lbs In good condition Has Linux on it. http://www.newegg.com/Product/Product.aspx?Item=N82E16834314193 -- Brian Ebenezer Enterprises http://webEbenezer.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrome at mn.real-time.com Tue Sep 8 14:41:52 2015 From: chrome at mn.real-time.com (Carl Soderstrom) Date: Tue, 8 Sep 2015 14:41:52 -0500 (CDT) Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: Message-ID: <1307786053.30148.1441741312073.JavaMail.root@mn.real-time.com> ----- Original Message ----- > From: "gregrwm" > BackupPC has been doing my own and my clients' backups for years now, > i'm not aware of anything that competes. I have to agree. I've been administrating BackupPC servers for over a decade now, and it's worked really well. The documentation is basically "read the config file" and if you do that, the comments in it explain everything pretty well. The only real downside is that it should be set up on a dedicated server (tho if you just give it its own storage disk that works, just doesn't protect against as much). It doesn't deal with multiple external hard drives as nicely as some backup scripts, but: a: one shouldn't fully trust external drives unless they're tended by a person who verifies that the drive is plugged in and the data went on it b: there's no harm in having a second backup method that sends the data to disk (even if it's just running rsync by hand). c: if you just use plain rsync to copy your data to the external drive, you don't have to worry about strange formats your data may be in. As for things which compete with BackupPC, Crashplan is pretty good. It's payware and occasionally buggy, so if you use it a support contract is worthwhile. However it's pretty powerful and featureful from a business backup perspective. -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com From tclug1 at whitleymott.net Tue Sep 8 17:56:57 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Tue, 8 Sep 2015 17:56:57 -0500 Subject: [tclug-list] t989 begging upgrade Message-ID: > > since firefox frequently grinds GINGERBREAD.UVLDG to a standstill, i want > to upgrade the OS on my galaxy sii sgh-t989. > > i've tried both Benjamin-Dobell/Heimdall and jamiejackson/Heimdall, either > can examine the phone partitions, and both seem to go thru the motions of > flashing the recovery partition according to the linux end, but the phone > doesn't show any blue progress bar, instead shows "Firmware upgrade > encountered an issue" (clearable with "close-pc-screen"). i think my usb > cable is fine because adb push works and the pushed files' md5/sha1 are > fine. > "ok, got it." well i posted the question, and finally found the answer, so i'll post it. bore warning, unless you're interested in upgrading the OS on a t989, or find amusement watching mice in mazes. i chose CM11 (kitkat), seeing reports that lollipop and other roms are not as good for battery life. getting there, well, in retrospect, simple enough, but wading through endless outdated or inapplicable or inaccurate even tho very well intended misinformation may not be the quest for the holy grail but was no piece of cake either. the cm t989 install instructions are pretty poor, clearly generated from a template and often misleading for the t989. even so they were indeed helpful and should be read carefully. maybe my summary here might help someone, like even myself if there's ever another goaround on this phone. first i tried fastboot. from my attempts, and from something i read, i ended up concluding it didn't support the t989. in retrospect i wonder if i was misled by the same circumstances confusing my attempts using heimdall. i was hoping not to have to compile heimdall from source, warnings abound about against using mainstream heimdall versions on a t989. i ended up compiling several versions of heimdall, both under vivid and under precise, looking for one that would work. following some cm wiki page for building heimdall was a mistake. the source tree Linux/README is good if ignoring all but Appendix B. and this page explains how to run heimdall&adb as a normal user. from my experience with my d710, not seeing a blue transfer bar on the phone led me to presume that meant heimdall flash wasn't working properly, plus i was getting "Firmware upgrade encountered an issue" on the phone, whereupon trying to reboot the phone would just return to that message. eventually i found that "heimdall close-pc-screen" clears that bootlockout, tho only when omitting "--no-reboot". jamiejackson/Heimdall did work on vivid, i was able to realize, once i let go of looking for a blue transfer bar on the phone, and stopped trying to flash CWM. trying to flash CWM always led to the "Firmware upgrade encountered an issue" bootlockout. perhaps a different version of CWM might have worked. TWRP-2.8.7.0 at least had the grace to boot itself up but couldn't find /sdcard. TWRP-2.8.5.0 worked. >apt-get install android-tools-adb >#tick t989 usb debugging, plug to pc >adb push SGH-T989_UVMC6_radio.zip /sdcard/ >adb push SGH-T989_UVMC6_radio.zip.md5 /sdcard/ >adb push gapps-kk-20140606-signed.zip /sdcard/ >adb push gapps-kk-20140606-signed.zip.md5 /sdcard/ >adb push cm-11-20150823-NIGHTLY-hercules.zip /sdcard/ >adb push cm-11-20150823-NIGHTLY-hercules.zip.sha1 /sdcard/ >adb shell ls -a -l /sdcard/|sort -k4 -n|listsort|&t >md5sum -c openrecovery-twrp-2.8.5.0-sgh-t989.img.md5 >#download mode(vol up&down & plug to pc) >Heimdall/heimdall/heimdall flash --recovery openrecovery-twrp-2.8.5.0-sgh-t989.img --no-reboot >adb reboot recovery #or vol up&power >#use TWRP to wipe, and install radio, cm11, &gapps first time i tried installing cm11, it barfed complaining my radio was incompatible, which tempted me to think perhaps one of the roms touted on galaxys2root might just include a compatible radio and be an easier way to go, but i oogled around, found SGH-T989_UVMC6_radio.zip and used it, even though i had to disable twrp's signature verification in order to use it. at least the md5 passed. twrp verifies using an .md5 file if present. md5's are usually provided alongside .zip's but usually not in the form of an .md5 file, tho the .md5 files are easy to make and provide a tad of comfort when flashing, especially when flashing a radio without sig verification. i turned signature verification back on for cm11&gapps . the sig verification checkbox in twrp's install dialog lead me to presume i could choose per-zip, but no, the choice is per install batch, so i installed the radio separately with it off, then cm11&gapps with it on. no idea if this repartitioning warning is applicable to cm11, but anyway i did flash it 3 times. and indeed so far it does seem worth the effort, cm11 isn't grinding to a standstill running firefox.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbmiller+l at gmail.com Wed Sep 9 12:10:08 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 9 Sep 2015 12:10:08 -0500 (CDT) Subject: [tclug-list] ntpd and ntpdate-debian Message-ID: For years I have been using something like this in a root crontab to adjust the time every 6 hours: 10 4,10,16,22 * * * /usr/sbin/ntpdate-debian On this machine it was going off by about 0.17 seconds every 6 hours and it was pretty consistent: 7 Sep 16:10:10 ntpdate[11932]: adjust time server 91.189.89.199 offset 0.172249 sec 7 Sep 22:10:09 ntpdate[13949]: adjust time server 91.189.89.199 offset 0.173074 sec 8 Sep 04:10:10 ntpdate[15490]: adjust time server 91.189.89.199 offset 0.174787 sec 8 Sep 10:10:09 ntpdate[18482]: adjust time server 91.189.89.199 offset 0.169422 sec Then I noticed that on a newer Ubuntu installation I didn't have the crontab, but the timing was even better. I'm pretty sure that newer Ubuntu installs let the user to choose to set date/time "automatically," and that was what I had chosen. So I had to wonder what it *really* was doing. I think it was running ntpd. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ntp 23926 0.0 0.0 39832 2264 ? Ss Sep08 0:01 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /var/lib/ntp/ntp.conf.dhcp -u 119:128 So I installed the ntp package like so: sudo apt-get install ntp That automatically set everything up and started it running. It can be called with the service command to ntp... sudo service ntp [start stop restart] ...which runs the script here: /etc/init.d/ntp That seems to keep the clock set very precisely. I guess the downside is that it is always running, but it if is using no more than 2.3 MB, that isn't a problem. Is this what everyone is doing these days? Mike From jeremy.mountainjohnson at gmail.com Wed Sep 9 12:43:46 2015 From: jeremy.mountainjohnson at gmail.com (Jeremy MountainJohnson) Date: Wed, 9 Sep 2015 12:43:46 -0500 Subject: [tclug-list] ntpd and ntpdate-debian In-Reply-To: References: Message-ID: I was using ntpdate (the smallest ntp client) with crontab for awhile too. This was mostly on an older NAS a few years ago. Now, I stick with running the daemon configured as a client only (for servers and workstations). As you correctly stated, it's a fairly small footprint, and the gain in precision makes it worth in most cases I can think of. Also have been running constantly on my rasp pis now for a few years (you almost have to with the lack of time retention on them). So, IMO, not much performance / memory gleaned using the minimal client like the old days (even then it was pretty minimal). You lose a little ground on security if not configured properly (ie. as a server on a lax network intended to be configured as a client only). -- Jeremy MountainJohnson Jeremy.MountainJohnson at gmail.com On Wed, Sep 9, 2015 at 12:10 PM, Mike Miller wrote: > For years I have been using something like this in a root crontab to adjust > the time every 6 hours: > > 10 4,10,16,22 * * * /usr/sbin/ntpdate-debian > > On this machine it was going off by about 0.17 seconds every 6 hours and it > was pretty consistent: > > 7 Sep 16:10:10 ntpdate[11932]: adjust time server 91.189.89.199 offset > 0.172249 sec > 7 Sep 22:10:09 ntpdate[13949]: adjust time server 91.189.89.199 offset > 0.173074 sec > 8 Sep 04:10:10 ntpdate[15490]: adjust time server 91.189.89.199 offset > 0.174787 sec > 8 Sep 10:10:09 ntpdate[18482]: adjust time server 91.189.89.199 offset > 0.169422 sec > > Then I noticed that on a newer Ubuntu installation I didn't have the > crontab, but the timing was even better. I'm pretty sure that newer Ubuntu > installs let the user to choose to set date/time "automatically," and that > was what I had chosen. > > So I had to wonder what it *really* was doing. I think it was running ntpd. > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND > ntp 23926 0.0 0.0 39832 2264 ? Ss Sep08 0:01 > /usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /var/lib/ntp/ntp.conf.dhcp -u > 119:128 > > So I installed the ntp package like so: > > sudo apt-get install ntp > > That automatically set everything up and started it running. It can be > called with the service command to ntp... > > sudo service ntp [start stop restart] > > ...which runs the script here: > > /etc/init.d/ntp > > That seems to keep the clock set very precisely. > > I guess the downside is that it is always running, but it if is using no > more than 2.3 MB, that isn't a problem. > > Is this what everyone is doing these days? > > Mike > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list From wendell.bell at gmail.com Wed Sep 9 13:10:35 2015 From: wendell.bell at gmail.com (Wendell Bell) Date: Wed, 09 Sep 2015 18:10:35 +0000 Subject: [tclug-list] t989 begging upgrade In-Reply-To: References: Message-ID: Does anyone here do these CyanogenMod installs for others for cash? The instructions on the Cyanogen wiki and then this inquiry convince me that undertaking this install by my lonesome would be FAR beyond my capabilities. Yet I have a perfectly good Asus Nexus 7 (2013) that Mountain View will no longer update. Or does anyone know of someone who does this mods competently? Thanks in advance! On Tue, Sep 8, 2015 at 5:58 PM gregrwm wrote: > since firefox frequently grinds GINGERBREAD.UVLDG to a standstill, i want >> to upgrade the OS on my galaxy sii sgh-t989. >> >> i've tried both Benjamin-Dobell/Heimdall and jamiejackson/Heimdall, >> either can examine the phone partitions, and both seem to go thru the >> motions of flashing the recovery partition according to the linux end, but >> the phone doesn't show any blue progress bar, instead shows "Firmware >> upgrade encountered an issue" (clearable with "close-pc-screen"). i think >> my usb cable is fine because adb push works and the pushed files' md5/sha1 >> are fine. >> > > "ok, got it." well i posted the question, and finally found the answer, > so i'll post it. bore warning, unless you're interested in upgrading the > OS on a t989, or find amusement watching mice in mazes. i chose CM11 > (kitkat), > seeing reports that lollipop and other roms are not as good for battery > life. getting there, well, in retrospect, simple enough, but wading > through endless outdated or inapplicable or inaccurate even tho very well > intended misinformation may not be the quest for the holy grail but was no > piece of cake either. the cm t989 install instructions > are pretty poor, > clearly generated from a template and often misleading for the t989. even > so they were indeed helpful and should be read carefully. maybe my summary > here might help someone, like even myself if there's ever another goaround > on this phone. > > first i tried fastboot. from my attempts, and from something i read, i > ended up concluding it didn't support the t989. in retrospect i wonder if > i was misled by the same circumstances confusing my attempts using > heimdall. i was hoping not to have to compile heimdall from source, > warnings abound about against using mainstream heimdall versions on a > t989. i ended up compiling several versions of heimdall, both under vivid > and under precise, looking for one that would work. following some cm wiki > page for building heimdall was a mistake. the source tree Linux/README is > good if ignoring all but Appendix B. and this page > explains how to run heimdall&adb as > a normal user. > > from my experience with my d710, not seeing a blue transfer bar on the > phone led me to presume that meant heimdall flash wasn't working properly, > plus i was getting "Firmware upgrade encountered an issue" on the phone, > whereupon trying to reboot the phone would just return to that message. > eventually i found that "heimdall close-pc-screen" clears that bootlockout, > tho only when omitting "--no-reboot". > > jamiejackson/Heimdall did > work on vivid, i was able to realize, once i let go of looking for a blue > transfer bar on the phone, and stopped trying to flash CWM. trying to > flash CWM always led to the "Firmware upgrade encountered an issue" > bootlockout. perhaps a different version of CWM might have worked. > TWRP-2.8.7.0 at least had the grace to boot itself up but couldn't find > /sdcard. TWRP-2.8.5.0 > > worked. > > >apt-get install android-tools-adb > >#tick t989 usb debugging, plug to pc > >adb push SGH-T989_UVMC6_radio.zip /sdcard/ > >adb push SGH-T989_UVMC6_radio.zip.md5 /sdcard/ > >adb push gapps-kk-20140606-signed.zip /sdcard/ > >adb push gapps-kk-20140606-signed.zip.md5 /sdcard/ > >adb push cm-11-20150823-NIGHTLY-hercules.zip /sdcard/ > >adb push cm-11-20150823-NIGHTLY-hercules.zip.sha1 /sdcard/ > >adb shell ls -a -l /sdcard/|sort -k4 -n|listsort|&t > >md5sum -c openrecovery-twrp-2.8.5.0-sgh-t989.img.md5 > >#download mode(vol up&down & plug to pc) > >Heimdall/heimdall/heimdall flash --recovery > openrecovery-twrp-2.8.5.0-sgh-t989.img --no-reboot > >adb reboot recovery #or vol up&power > >#use TWRP to wipe, and install radio, cm11, &gapps > > first time i tried installing cm11, it barfed complaining my radio was > incompatible, which tempted me to think perhaps one of the roms touted on > galaxys2root might just include a compatible radio and be an easier way to > go, but i oogled around, found SGH-T989_UVMC6_radio.zip and used it, even > though i had to disable twrp's signature verification in order to use it. > at least the md5 passed. twrp verifies using an .md5 file if present. > md5's are usually provided alongside .zip's but usually not in the form of > an .md5 file, tho the .md5 files are easy to make and provide a tad of > comfort when flashing, especially when flashing a radio without sig > verification. i turned signature verification back on for cm11&gapps > . the sig > verification checkbox in twrp's install dialog lead me to presume i could > choose per-zip, but no, the choice is per install batch, so i installed the > radio separately with it off, then cm11&gapps with it on. no idea if this > repartitioning warning > > is applicable to cm11, but anyway i did flash it 3 times. > > and indeed so far it does seem worth the effort, cm11 isn't grinding to a > standstill running firefox.. > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew at lunn.ch Wed Sep 9 12:35:23 2015 From: andrew at lunn.ch (Andrew Lunn) Date: Wed, 9 Sep 2015 19:35:23 +0200 Subject: [tclug-list] ntpd and ntpdate-debian In-Reply-To: References: Message-ID: <20150909173523.GQ29184@lunn.ch> > Is this what everyone is doing these days? Pretty much. Although i tend to use openntpd, rather than ntp. Being an openbsd project, it might have better network security. You might want to check the man page for -g, if you are used to using ntp-date. Andrew From mbmiller+l at gmail.com Wed Sep 9 13:25:19 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 9 Sep 2015 13:25:19 -0500 (CDT) Subject: [tclug-list] ntpd and ntpdate-debian In-Reply-To: References: Message-ID: Thanks, Jeremy. I guess I'm not running as a service, so I'm probably fairly safe. I did just notice that the ntpd is using /etc/ntp.conf and it has this info about ntp servers: -----begin ntp.conf excerpt on next line--------- # Specify one or more NTP servers. # Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board # on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for # more information. server 0.ubuntu.pool.ntp.org server 1.ubuntu.pool.ntp.org server 2.ubuntu.pool.ntp.org server 3.ubuntu.pool.ntp.org # Use Ubuntu's ntp server as a fallback. server ntp.ubuntu.com -----end ntp.conf excerpt on previous line--------- This machine is at umn.edu, so I should probably add the local time servers, which I guess are these: server 128.101.101.101 server 134.84.84.84 (a.k.a. ntp.umn.edu; Same as the nameservers in /etc/resolv.conf) Mike On Wed, 9 Sep 2015, Jeremy MountainJohnson wrote: > I was using ntpdate (the smallest ntp client) with crontab for awhile > too. This was mostly on an older NAS a few years ago. Now, I stick > with running the daemon configured as a client only (for servers and > workstations). As you correctly stated, it's a fairly small footprint, > and the gain in precision makes it worth in most cases I can think of. > Also have been running constantly on my rasp pis now for a few years > (you almost have to with the lack of time retention on them). > > So, IMO, not much performance / memory gleaned using the minimal > client like the old days (even then it was pretty minimal). You lose a > little ground on security if not configured properly (ie. as a server > on a lax network intended to be configured as a client only). > > -- > Jeremy MountainJohnson > Jeremy.MountainJohnson at gmail.com > > > On Wed, Sep 9, 2015 at 12:10 PM, Mike Miller wrote: >> For years I have been using something like this in a root crontab to adjust >> the time every 6 hours: >> >> 10 4,10,16,22 * * * /usr/sbin/ntpdate-debian >> >> On this machine it was going off by about 0.17 seconds every 6 hours and it >> was pretty consistent: >> >> 7 Sep 16:10:10 ntpdate[11932]: adjust time server 91.189.89.199 offset >> 0.172249 sec >> 7 Sep 22:10:09 ntpdate[13949]: adjust time server 91.189.89.199 offset >> 0.173074 sec >> 8 Sep 04:10:10 ntpdate[15490]: adjust time server 91.189.89.199 offset >> 0.174787 sec >> 8 Sep 10:10:09 ntpdate[18482]: adjust time server 91.189.89.199 offset >> 0.169422 sec >> >> Then I noticed that on a newer Ubuntu installation I didn't have the >> crontab, but the timing was even better. I'm pretty sure that newer Ubuntu >> installs let the user to choose to set date/time "automatically," and that >> was what I had chosen. >> >> So I had to wonder what it *really* was doing. I think it was running ntpd. >> >> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >> ntp 23926 0.0 0.0 39832 2264 ? Ss Sep08 0:01 >> /usr/sbin/ntpd -p /var/run/ntpd.pid -g -c /var/lib/ntp/ntp.conf.dhcp -u >> 119:128 >> >> So I installed the ntp package like so: >> >> sudo apt-get install ntp >> >> That automatically set everything up and started it running. It can be >> called with the service command to ntp... >> >> sudo service ntp [start stop restart] >> >> ...which runs the script here: >> >> /etc/init.d/ntp >> >> That seems to keep the clock set very precisely. >> >> I guess the downside is that it is always running, but it if is using no >> more than 2.3 MB, that isn't a problem. >> >> Is this what everyone is doing these days? >> >> Mike >> _______________________________________________ >> TCLUG Mailing List - Minneapolis/St. Paul, Minnesota >> tclug-list at mn-linux.org >> http://mailman.mn-linux.org/mailman/listinfo/tclug-list > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > From mbmiller+l at gmail.com Wed Sep 9 14:46:51 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 9 Sep 2015 14:46:51 -0500 (CDT) Subject: [tclug-list] ntpd and ntpdate-debian In-Reply-To: <20150909173523.GQ29184@lunn.ch> References: <20150909173523.GQ29184@lunn.ch> Message-ID: On Wed, 9 Sep 2015, Andrew Lunn wrote: >> Is this what everyone is doing these days? > > Pretty much. Although i tend to use openntpd, rather than ntp. Being an > openbsd project, it might have better network security. > > You might want to check the man page for -g, if you are used to using > ntp-date. In "man ntpd", the -g option leads me to the -q which has this info: -q Exit the ntpd just after the first time the clock is set. This behavior mimics that of the ntpdate program, which is to be retired. The -g and -x options can be used with this option. So they are putting ntpdate out to pasture and ntpd is taking over. Mike From tclug1 at whitleymott.net Wed Sep 9 17:24:03 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Wed, 9 Sep 2015 17:24:03 -0500 Subject: [tclug-list] t989 begging upgrade In-Reply-To: References: Message-ID: i've upgraded both my d710 and my t989 now, and i "kept it simple" by removing everything of interest from the phone and not bothering with backups. if you don't find anyone with more direct experience i'll be glad to help, no guarantee of outcome, pay neither required nor refused.. On 9 September 2015 at 13:10, Wendell Bell wrote: > Does anyone here do these CyanogenMod installs for others for cash? The > instructions on the Cyanogen wiki and then this inquiry convince me that > undertaking this install by my lonesome would be FAR beyond my > capabilities. Yet I have a perfectly good Asus Nexus 7 (2013) that Mountain > View will no longer update. Or does anyone know of someone who does this > mods competently? Thanks in advance! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cncole at earthlink.net Wed Sep 9 19:55:54 2015 From: cncole at earthlink.net (Chuck Cole) Date: Wed, 9 Sep 2015 19:55:54 -0500 Subject: [tclug-list] t989 begging upgrade In-Reply-To: References: Message-ID: What can you do with the D710 after WIMAX cell technology is shut down by Sprint at the end of the month? Non-phone should be fine, but Sprint cellular won't work except for phones with 4G LTE chips which the D710 doesn't have. Will other carriers still support WIMAX? Chuck _____ From: tclug-list-bounces at mn-linux.org [mailto:tclug-list-bounces at mn-linux.org] On Behalf Of gregrwm Sent: Wednesday, September 09, 2015 5:24 PM To: tclug-list at mn-linux.org Subject: Re: [tclug-list] t989 begging upgrade i've upgraded both my d710 and my t989 now, and i "kept it simple" by removing everything of interest from the phone and not bothering with backups. if you don't find anyone with more direct experience i'll be glad to help, no guarantee of outcome, pay neither required nor refused.. On 9 September 2015 at 13:10, Wendell Bell wrote: Does anyone here do these CyanogenMod installs for others for cash? The instructions on the Cyanogen wiki and then this inquiry convince me that undertaking this install by my lonesome would be FAR beyond my capabilities. Yet I have a perfectly good Asus Nexus 7 (2013) that Mountain View will no longer update. Or does anyone know of someone who does this mods competently? Thanks in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tclug1 at whitleymott.net Thu Sep 10 12:27:07 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Thu, 10 Sep 2015 12:27:07 -0500 Subject: [tclug-list] t989 begging upgrade Message-ID: bother. wifi meets most my needs, even on the road. i made it through most of my life with no cellphone, and may do so some more. or might actually throw down some money for something. On 9 September 2015 at 19:55, Chuck Cole wrote: > What can you do with the D710 after WIMAX cell technology is shut down by > Sprint at the end of the month? > Non-phone should be fine, but Sprint cellular won't work except for phones > with 4G LTE chips which the D710 doesn't have. > Will other carriers still support WIMAX? > > Chuck > > ------------------------------ > *From:* tclug-list-bounces at mn-linux.org [mailto: > tclug-list-bounces at mn-linux.org] *On Behalf Of *gregrwm > *Sent:* Wednesday, September 09, 2015 5:24 PM > *To:* tclug-list at mn-linux.org > *Subject:* Re: [tclug-list] t989 begging upgrade > > i've upgraded both my d710 and my t989 now, and i "kept it simple" by > removing everything of interest from the phone and not bothering with > backups. if you don't find anyone with more direct experience i'll be glad > to help, no guarantee of outcome, pay neither required nor refused.. > > > On 9 September 2015 at 13:10, Wendell Bell wrote: > >> Does anyone here do these CyanogenMod installs for others for cash? The >> instructions on the Cyanogen wiki and then this inquiry convince me that >> undertaking this install by my lonesome would be FAR beyond my >> capabilities. Yet I have a perfectly good Asus Nexus 7 (2013) that Mountain >> View will no longer update. Or does anyone know of someone who does this >> mods competently? Thanks in advance! >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tclug1 at whitleymott.net Thu Sep 10 12:52:00 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Thu, 10 Sep 2015 12:52:00 -0500 Subject: [tclug-list] t989 begging upgrade Message-ID: no bother actually. a bit of oogling says it looks like i'll lose 4g. that's it. and i've left 4g shutoff virtually always anyway. 3g is fine. so i won't even notice. On 10 September 2015 at 12:27, gregrwm wrote: > bother. wifi meets most my needs, even on the road. i made it through > most of my life with no cellphone, and may do so some more. or might > actually throw down some money for something. > > > On 9 September 2015 at 19:55, Chuck Cole wrote: > >> What can you do with the D710 after WIMAX cell technology is shut down by >> Sprint at the end of the month? >> Non-phone should be fine, but Sprint cellular won't work except for >> phones with 4G LTE chips which the D710 doesn't have. >> Will other carriers still support WIMAX? >> >> Chuck >> >> ------------------------------ >> *From:* tclug-list-bounces at mn-linux.org [mailto: >> tclug-list-bounces at mn-linux.org] *On Behalf Of *gregrwm >> *Sent:* Wednesday, September 09, 2015 5:24 PM >> *To:* tclug-list at mn-linux.org >> *Subject:* Re: [tclug-list] t989 begging upgrade >> >> i've upgraded both my d710 and my t989 now, and i "kept it simple" by >> removing everything of interest from the phone and not bothering with >> backups. if you don't find anyone with more direct experience i'll be glad >> to help, no guarantee of outcome, pay neither required nor refused.. >> >> >> On 9 September 2015 at 13:10, Wendell Bell >> wrote: >> >>> Does anyone here do these CyanogenMod installs for others for cash? The >>> instructions on the Cyanogen wiki and then this inquiry convince me that >>> undertaking this install by my lonesome would be FAR beyond my >>> capabilities. Yet I have a perfectly good Asus Nexus 7 (2013) that Mountain >>> View will no longer update. Or does anyone know of someone who does this >>> mods competently? Thanks in advance! >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From cncole at earthlink.net Thu Sep 10 21:37:25 2015 From: cncole at earthlink.net (Chuck Cole) Date: Thu, 10 Sep 2015 21:37:25 -0500 Subject: [tclug-list] t989 begging upgrade In-Reply-To: References: Message-ID: <07D197FAB70C40E9B78C14696727C5DD@D830B> Good to hear that that you won't be cut off. I use the 4G often for various apps and sometimes as internet access for my PC. 4G LTE is much better than my 4G WIMAX was. I upgraded from my S2 to an S5 which has more and better chips in it, including a better bluetooth spec and GSM also. The S5 is very much better at everything. I'm only doing regular stuff tho. Using a Samsung microSD card seems to have fixed the camera anomaly I had with Micro Center cards. _____ From: tclug-list-bounces at mn-linux.org [mailto:tclug-list-bounces at mn-linux.org] On Behalf Of gregrwm Sent: Thursday, September 10, 2015 12:52 PM To: tclug-list at mn-linux.org Subject: Re: [tclug-list] t989 begging upgrade no bother actually. a bit of oogling says it looks like i'll lose 4g. that's it. and i've left 4g shutoff virtually always anyway. 3g is fine. so i won't even notice. On 10 September 2015 at 12:27, gregrwm wrote: bother. wifi meets most my needs, even on the road. i made it through most of my life with no cellphone, and may do so some more. or might actually throw down some money for something. On 9 September 2015 at 19:55, Chuck Cole wrote: What can you do with the D710 after WIMAX cell technology is shut down by Sprint at the end of the month? Non-phone should be fine, but Sprint cellular won't work except for phones with 4G LTE chips which the D710 doesn't have. Will other carriers still support WIMAX? Chuck _____ From: tclug-list-bounces at mn-linux.org [mailto:tclug-list-bounces at mn-linux.org] On Behalf Of gregrwm Sent: Wednesday, September 09, 2015 5:24 PM To: tclug-list at mn-linux.org Subject: Re: [tclug-list] t989 begging upgrade i've upgraded both my d710 and my t989 now, and i "kept it simple" by removing everything of interest from the phone and not bothering with backups. if you don't find anyone with more direct experience i'll be glad to help, no guarantee of outcome, pay neither required nor refused.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From n0nas at amsat.org Thu Sep 10 22:00:53 2015 From: n0nas at amsat.org (Doug Reed) Date: Thu, 10 Sep 2015 22:00:53 -0500 Subject: [tclug-list] t989 begging upgrade Message-ID: When Sprint turns off the Clearwire WiMax network on Nov 6, that is the end of it. Nobody else is using WiMax that I'm aware of, LTE won the 4G technology war. I'm using the WiMax network for Internet service, so I'm interested in the problem. My service provider will be selling me a new 4G LTE modem and I'm waiting to see how soon I can get it. Regarding using the old phone for telco service, it would probably keep working but only on the 3G network. At least that is my guess. My opinion is that Sprint 3G coverage isn't as good as 4G LTE. You could probably use it at home or anywhere else you can get WiFi service. Worst case, install Google Voice or Skype, turn off the mobile network, and use it strictly via the WiFi. Personally, I can find numerous things to use a smart phone for even if it doesn't have phone service. All that CPU power can be very useful. If you don't want it, I'll take it. :-) Doug Reed. N St Paul. From daniel.armbrust.list at gmail.com Thu Sep 10 23:05:02 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Thu, 10 Sep 2015 23:05:02 -0500 Subject: [tclug-list] Dual booting with Windows 8.1. What do the default partitions mean? In-Reply-To: <55E86FE2.2080307@gmail.com> References: <55E86FE2.2080307@gmail.com> Message-ID: <55F252EE.7050806@gmail.com> On 09/03/2015 11:05 AM, rhubarbpieguy at gmail.com wrote: > > > * > > free space 1 MB > > * > > /dev/sda1 ntfs 1073 MB > > * > > /dev/sda2 efi 104 MB > > * > > /dev/sda3 ntfs 134 MB > > * > > /dev/sda4 ntfs 489243 MB > > * > > /dev/sda5 ntfs 9550 MB > > * > > free space 0 MB > > > I understand sda2 and assume sda4 is the main Windows partition. But what were sda3 and > sda5? Can I delete them? Can I resize sda4 during my Linux installation? > > I've not booted to Windows since 2002 and probably still won't after this experiment. > But I'd be interested in Linux users' input. > I think Windows makes a boot partition of its own these days - that is likely sda3 - I wouldn't delete that. Its tiny anyway. sda5 is likely a restoration partition of sorts - many system builders include a partition with a windows image on it, that can be used to rebuild windows from scratch if it goes south. Boot windows, and look in the partition manager it has - it will tell you what each of them are. From daniel.armbrust.list at gmail.com Thu Sep 10 23:12:20 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Thu, 10 Sep 2015 23:12:20 -0500 Subject: [tclug-list] old ram gathering dust? Message-ID: <55F254A4.4010304@gmail.com> Anyone have any DDR era RAM gathering dust they don't need? I have a system that has PC2700 / 333Mhz DDR RAM in it, some of which randomly failed. For various reasons, in the short term, I just want the box back running again with more than the remaining 512MB of RAM. Unfortunately, I have older era RAM laying around, and newer era... but nothing that matches. I think this was from that annoying time frame when they kept changing the pin counts on RAM every other year or so. Thanks, Dan From kellywilliams81 at gmail.com Thu Sep 10 23:17:38 2015 From: kellywilliams81 at gmail.com (Kelly Williams) Date: Thu, 10 Sep 2015 23:17:38 -0500 Subject: [tclug-list] old ram gathering dust? In-Reply-To: <55F254A4.4010304@gmail.com> References: <55F254A4.4010304@gmail.com> Message-ID: Thats funny I am getting rid of a motheboard that has DDR1 ram in it I have 256, 512, 1gb sticks and I think they are 2700's... But I am no were close to the twin cities, I am down in the corner of Northwest Iowa in a little town call Sheldon.. We can work something out plus I can make my wife happy to by getting rid of some more junk that has been laying around.. Kelly On Thu, Sep 10, 2015 at 11:12 PM, Dan Armbrust < daniel.armbrust.list at gmail.com> wrote: > Anyone have any DDR era RAM gathering dust they don't need? > > I have a system that has PC2700 / 333Mhz DDR RAM in it, some of which > randomly failed. > > For various reasons, in the short term, I just want the box back running > again with more than the remaining 512MB of RAM. > > Unfortunately, I have older era RAM laying around, and newer era... but > nothing that matches. I think this was from that annoying time frame when > they kept changing the pin counts on RAM every other year or so. > > > Thanks, > > Dan > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nassarmu at gmail.com Fri Sep 11 09:02:00 2015 From: nassarmu at gmail.com (Munir Nassar) Date: Fri, 11 Sep 2015 09:02:00 -0500 Subject: [tclug-list] old ram gathering dust? In-Reply-To: <55F254A4.4010304@gmail.com> References: <55F254A4.4010304@gmail.com> Message-ID: Free Geek Twin Cities has all sorts of old RAM about. You can always look there for an exact match. On Thu, Sep 10, 2015 at 11:12 PM, Dan Armbrust wrote: > Anyone have any DDR era RAM gathering dust they don't need? > > I have a system that has PC2700 / 333Mhz DDR RAM in it, some of which > randomly failed. > > For various reasons, in the short term, I just want the box back running > again with more than the remaining 512MB of RAM. > > Unfortunately, I have older era RAM laying around, and newer era... but > nothing that matches. I think this was from that annoying time frame when > they kept changing the pin counts on RAM every other year or so. > > > Thanks, > > Dan > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list From nassarmu at gmail.com Fri Sep 11 09:06:32 2015 From: nassarmu at gmail.com (Munir Nassar) Date: Fri, 11 Sep 2015 09:06:32 -0500 Subject: [tclug-list] Dual booting with Windows 8.1. What do the default partitions mean? In-Reply-To: <55E86FE2.2080307@gmail.com> References: <55E86FE2.2080307@gmail.com> Message-ID: I think you are confusing EFI(bios replacement) with GPT(MBR partition table replacement), GPT is required to use EFI but not the other way around. So you can use GPT and the large(not unlimited) partitions it offers and switch back to BIOS. The caveat is that if you do so, there is a very good chance that you will confuse grub to no end and the system will not boot without additional laying of hands, grub reinstallation, etc. and so forth. On Thu, Sep 3, 2015 at 11:05 AM, wrote: > > > I recently purchased a new box on which I wiped the drive and installed > Linux using efi. My efi partition is sda1, with Linux distributions on > sda[2-3] and sda4 as swap. However, if I don't have Windows, is there an > advantage of using efi? If I understand, I can enable legacy BIOS and lose > the efi partition. In that case I could have Linux distributions on > sda[1-3]. I believe efi has effectively no partition limitations, so I > could have efi on sda1, Linux distributions on sda[2-3] and sda5 as swap. > But I don't see the advantage of efi as four partitions is enough for me. > > Also, while I'll probably keep my current setup, I realize the danger of not > trying new things. I'd like to attempt dual booting with Windows but am > confused. The default partitions were: > > * > > free space 1 MB > > * > > /dev/sda1 ntfs 1073 MB > > * > > /dev/sda2 efi 104 MB > > * > > /dev/sda3 ntfs 134 MB > > * > > /dev/sda4 ntfs 489243 MB > > * > > /dev/sda5 ntfs 9550 MB > > * > > free space 0 MB > > > I understand sda2 and assume sda4 is the main Windows partition. But what > were sda3 and sda5? Can I delete them? Can I resize sda4 during my Linux > installation? > > I've not booted to Windows since 2002 and probably still won't after this > experiment. But I'd be interested in Linux users' input. > > > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list From daniel.armbrust.list at gmail.com Fri Sep 11 15:52:11 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Fri, 11 Sep 2015 15:52:11 -0500 Subject: [tclug-list] old ram gathering dust? In-Reply-To: References: <55F254A4.4010304@gmail.com> Message-ID: <55F33EFB.3090307@gmail.com> Cool - never been there before. Maybe I should gather some stuff up and see if they trade :) Thanks, dan On 09/11/2015 09:02 AM, Munir Nassar wrote: > Free Geek Twin Cities has all sorts of old RAM about. You can always > look there for an exact match. > > On Thu, Sep 10, 2015 at 11:12 PM, Dan Armbrust > wrote: >> Anyone have any DDR era RAM gathering dust they don't need? >> >> I have a system that has PC2700 / 333Mhz DDR RAM in it, some of which >> randomly failed. >> >> For various reasons, in the short term, I just want the box back running >> again with more than the remaining 512MB of RAM. >> >> Unfortunately, I have older era RAM laying around, and newer era... but >> nothing that matches. I think this was from that annoying time frame when >> they kept changing the pin counts on RAM every other year or so. >> >> From mbmiller+l at gmail.com Fri Sep 11 23:56:46 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Fri, 11 Sep 2015 23:56:46 -0500 (CDT) Subject: [tclug-list] recovering data from failing ext4 drive Message-ID: I have an internal hard drive in ext4 format that failed. There are a few things I'd like to recover from it. It wouldn't boot, but I thought maybe I could mount it. No such luck. In the text below, lines beginning with a dollar sign prompt are commands. There are three commands followed by output. Apparently, the dmesg output might be useful, but I have no idea what it means. Do any of you have experience with this kind of problem? Is there anything I might try next? TIA, Mike $ sudo parted -l /dev/sda | head > shit Model: ATA ST32000542AS (scsi) Disk /dev/sda: 2000GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 1049kB 1992GB 1992GB primary ext4 raid $ sudo mount -t ext4 /dev/sda1 /media/mbmiller/a mount: wrong fs type, bad option, bad superblock on /dev/sda1, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. $ dmesg | tail -n19 [168330.232384] sda: sda1 [169448.936028] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 [169448.936033] ata3.00: BMDMA stat 0x24 [169448.936037] ata3.00: failed command: READ DMA EXT [169448.936042] ata3.00: cmd 25/00:f8:58:1d:c5/00:00:73:00:00/e0 tag 0 dma 126976 in res 51/40:00:1a:1e:c5/40:00:73:00:00/00 Emask 0x9 (media error) [169448.936045] ata3.00: status: { DRDY ERR } [169448.936046] ata3.00: error: { UNC } [169449.048405] ata3.00: configured for UDMA/133 [169449.048439] sd 2:0:0:0: [sda] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [169449.048442] sd 2:0:0:0: [sda] Sense Key : Medium Error [current] [descriptor] [169449.048445] sd 2:0:0:0: [sda] Add. Sense: Unrecovered read error - auto reallocate failed [169449.048447] sd 2:0:0:0: [sda] CDB: [169449.048449] Read(10): 28 00 73 c5 1d 58 00 00 f8 00 [169449.048456] blk_update_request: I/O error, dev sda, sector 1942298138 [169449.048470] ata3: EH complete [169449.048603] JBD2: Failed to read block at offset 8899 [169449.048900] JBD2: recovery failed [169449.048903] EXT4-fs (sda1): error loading journal From daniel.armbrust.list at gmail.com Sat Sep 12 00:01:15 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Sat, 12 Sep 2015 00:01:15 -0500 Subject: [tclug-list] old ram gathering dust? In-Reply-To: References: <55F254A4.4010304@gmail.com> Message-ID: <55F3B19B.6060008@gmail.com> Hi Kelly, I ended up going over to Free Geek this afternoon and picking some up, as I needed to get the machine back online tomorrow. Someday, I'll have time to move the drives around, retire this in place of an only slightly out of date computer, instead of completely out of date... DDR ram is the oldest generation of RAM they keep at Free Geek, however, everything older goes in the melt pile (so I was told). Thanks, Dan On 09/10/2015 11:17 PM, Kelly Williams wrote: > Thats funny I am getting rid of a motheboard that has DDR1 ram in it I have 256, 512, > 1gb sticks and I think they are 2700's... But I am no were close to the twin cities, I > am down in the corner of Northwest Iowa in a little town call Sheldon.. We can work > something out plus I can make my wife happy to by getting rid of some more junk that has > been laying around.. > > Kelly > From daniel.armbrust.list at gmail.com Sat Sep 12 00:10:08 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Sat, 12 Sep 2015 00:10:08 -0500 Subject: [tclug-list] formatting an external drive with ext4 for backups In-Reply-To: References: <55EA029E.6020707@kateley.com> Message-ID: <55F3B3B0.5020503@gmail.com> A couple of thoughts: Of any File system I've used in recent years, ext4 is the _only_ one that has caused me data loss. And serious data loss. I've used either the older reiser, or XFS, on big drives for backup stuff without any issue. Not sure what the current crop of "good" file systems are... but I've never considered ext4 one of them since it failed its most basic functions for me. If these drives will be travelling about - I would encrypt the entire drive partition with something like truecrypt (or whatever current supported encryption scheme is out there... shame what happened to truecrypt...) - and then put your file system of choice inside that. If the drive is lost / stolen, it will just appear as an unformatted disk. Dan On 09/04/2015 04:40 PM, Mike Miller wrote: > I have the impression that zfs is a safer and more reliable system than ext4, so maybe I > should want zfs, but is it working on Linux? Is it hard to get it installed for use > with an external drive? > > I guess $0.06/GB-month is a good deal if you have a small amount of data to backup. > > Mike > From daniel.armbrust.list at gmail.com Sat Sep 12 09:11:41 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Sat, 12 Sep 2015 09:11:41 -0500 Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: References: Message-ID: <55F4329D.2030408@gmail.com> On 09/11/2015 11:56 PM, Mike Miller wrote: > > [169448.936046] ata3.00: error: { UNC } > [169449.048405] ata3.00: configured for UDMA/133 > [169449.048439] sd 2:0:0:0: [sda] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE > [169449.048442] sd 2:0:0:0: [sda] Sense Key : Medium Error [current] [descriptor] > [169449.048445] sd 2:0:0:0: [sda] Add. Sense: Unrecovered read error - auto reallocate > failed > [169449.048447] sd 2:0:0:0: [sda] CDB: [169449.048449] Read(10): 28 00 73 c5 1d 58 00 00 > f8 00 > [169449.048456] blk_update_request: I/O error, dev sda, sector 1942298138 > [169449.048470] ata3: EH complete > [169449.048603] JBD2: Failed to read block at offset 8899 > [169449.048900] JBD2: recovery failed > [169449.048903] EXT4-fs (sda1): error loading journal > I just wrote you a different response last night, where I noted that ext4 is the only one that cost me data... and here again, a single IO error on your drive, and ext4 has puked. It used to be that you could mount ext4 as ext3 - which would cause it to ignore the journal - but I think they have since changed the format of ext4, and that is no longer possible (unless you created this ext4 a really long time ago, when they were still compatible, by default) As far as trying anything else for recovery - the first step is to get whatever is on this drive onto good hardware - as any further operations may lead to more damaged blocks, making the problem even worse. Use a tool like one of these, to copy the entire disk, bit for bit onto that nice, new empty drive that you have: http://askubuntu.com/questions/211578/whats-the-difference-between-ddrescue-gddrescue-and-dd-rescue Its possible, that after copying the entire disk, bit for bit - that you might be able to mount it from the new drive - because the rescue tools do a much better job of isolating bad bits (as opposed to failing an entire sector read), and perhaps the remaining errors will be small enough to let ext4 complete the mount. If not, start going down one of these recovery paths: https://linuxexpresso.wordpress.com/2010/03/31/repair-a-broken-ext4-superblock-in-ubuntu/ But ONLY on the new drive - after you copied the data. The failing drive could completely disappear on you at any time. After you get done doing all of this, and hopefully get back the data you want - then you can go down the road of reformatting that new disk the way you want it in the end. From your other message: > I think parted is pretty easy to use, once you know what to do. I guess I need to use > parted -l to find the /dev/sd letter (I'll use /dev/sdx here), then something like this > will create the ext4 partition: > > sudo parted /dev/sdx mklabel gpt > sudo parted /dev/sdx mkpart primary ext4 0 5TiB > > I think it then needs to be formatted. Would this command do it? > > mkfs.ext4 /dev/sdx1 This sounds about right - but all of this will be much easier if you just use gparted. Also, I'd still recommend doing an encryption step. So, after you make the primary partition, with truecrypt, anyway, you encrypt the entire primary partition. Then, after you "mount" the encrypted partition with truecrypt, you do the file system create steps inside of that encrypted container. I'm sure other encryption tools are similar, I'm just not familiar with them. You can find the last good version of truecrypt here: https://www.grc.com/misc/truecrypt/truecrypt.htm including the user guide: https://www.grc.com/misc/truecrypt/TrueCrypt%20User%20Guide.pdf From mbmiller+l at gmail.com Sat Sep 12 15:21:48 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Sat, 12 Sep 2015 15:21:48 -0500 (CDT) Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: <55F4329D.2030408@gmail.com> References: <55F4329D.2030408@gmail.com> Message-ID: Wow, Dan, thanks so much for all the ideas! This is a huge help. Here's what's going on: I already formatted the new drive with ext4 and copied 3 TB of data onto it, so I don't want to undo all that right away, but there is another, probably more appealing way to go: /dev/sda and /dev/sdb are identical 2 TB drives that were previously in a RAID1. It looks like sdb somehow disconnected from the RAID, but sda kept working for a few months before it failed. I can mount /dev/sdb1 just fine and I copied all the files off of it onto the new external drive. I had no errors. So maybe /dev/sdb is in pretty good shape. Now that it's backed up, maybe the best plan is to try to copy /dev/sda to /dev/sdb using one of the dd tools. So here's a question: /dev/sdb is formatted for ext4. If I want to use it as the destination drive for the dd copy, do I have to use parted to remove the partition table first? Or what? Thanks again! Mike On Sat, 12 Sep 2015, Dan Armbrust wrote: > On 09/11/2015 11:56 PM, Mike Miller wrote: >> >> [169448.936046] ata3.00: error: { UNC } >> [169449.048405] ata3.00: configured for UDMA/133 >> [169449.048439] sd 2:0:0:0: [sda] FAILED Result: hostbyte=DID_OK >> driverbyte=DRIVER_SENSE >> [169449.048442] sd 2:0:0:0: [sda] Sense Key : Medium Error [current] >> [descriptor] >> [169449.048445] sd 2:0:0:0: [sda] Add. Sense: Unrecovered read error - auto >> reallocate failed >> [169449.048447] sd 2:0:0:0: [sda] CDB: [169449.048449] Read(10): 28 00 73 >> c5 1d 58 00 00 f8 00 >> [169449.048456] blk_update_request: I/O error, dev sda, sector 1942298138 >> [169449.048470] ata3: EH complete >> [169449.048603] JBD2: Failed to read block at offset 8899 >> [169449.048900] JBD2: recovery failed >> [169449.048903] EXT4-fs (sda1): error loading journal >> > > I just wrote you a different response last night, where I noted that ext4 is > the only one that cost me data... and here again, a single IO error on your > drive, and ext4 has puked. > > It used to be that you could mount ext4 as ext3 - which would cause it to > ignore the journal - but I think they have since changed the format of ext4, > and that is no longer possible (unless you created this ext4 a really long > time ago, when they were still compatible, by default) > > As far as trying anything else for recovery - the first step is to get > whatever is on this drive onto good hardware - as any further operations may > lead to more damaged blocks, making the problem even worse. > > Use a tool like one of these, to copy the entire disk, bit for bit onto that > nice, new empty drive that you have: > > http://askubuntu.com/questions/211578/whats-the-difference-between-ddrescue-gddrescue-and-dd-rescue > > > Its possible, that after copying the entire disk, bit for bit - that you > might be able to mount it from the new drive - because the rescue tools do a > much better job of isolating bad bits (as opposed to failing an entire sector > read), and perhaps the remaining errors will be small enough to let ext4 > complete the mount. > > If not, start going down one of these recovery paths: > https://linuxexpresso.wordpress.com/2010/03/31/repair-a-broken-ext4-superblock-in-ubuntu/ > > But ONLY on the new drive - after you copied the data. The failing drive > could completely disappear on you at any time. > > After you get done doing all of this, and hopefully get back the data you > want - then you can go down the road of reformatting that new disk the way > you want it in the end. > > From your other message: > >> I think parted is pretty easy to use, once you know what to do. I guess I >> need to use parted -l to find the /dev/sd letter (I'll use /dev/sdx here), >> then something like this will create the ext4 partition: >> >> sudo parted /dev/sdx mklabel gpt >> sudo parted /dev/sdx mkpart primary ext4 0 5TiB >> >> I think it then needs to be formatted. Would this command do it? >> >> mkfs.ext4 /dev/sdx1 > > This sounds about right - but all of this will be much easier if you just use > gparted. > > Also, I'd still recommend doing an encryption step. So, after you make the > primary partition, with truecrypt, anyway, you encrypt the entire primary > partition. Then, after you "mount" the encrypted partition with truecrypt, > you do the file system create steps inside of that encrypted container. > > I'm sure other encryption tools are similar, I'm just not familiar with them. > > You can find the last good version of truecrypt here: > https://www.grc.com/misc/truecrypt/truecrypt.htm including the user guide: > https://www.grc.com/misc/truecrypt/TrueCrypt%20User%20Guide.pdf > > > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > From daniel.armbrust.list at gmail.com Sun Sep 13 10:17:21 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Sun, 13 Sep 2015 10:17:21 -0500 Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: References: <55F4329D.2030408@gmail.com> Message-ID: <55F59381.6050401@gmail.com> An HTML attachment was scrubbed... URL: From mbmiller+l at gmail.com Sun Sep 13 11:19:29 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Sun, 13 Sep 2015 11:19:29 -0500 (CDT) Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: <55F59381.6050401@gmail.com> References: <55F4329D.2030408@gmail.com> <55F59381.6050401@gmail.com> Message-ID: Thanks again, Dan. I ended up starting it before getting your message below, but I think I've got it right. /dev/sdb1 was working and I backed up all of it. /dev/sda1 couldn't be mounted, but they previously were cloned. So I unmounted /dev/sdb, installed GNU ddrescue (on a third drive) and did this: sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt After that finishes, I will run this to try to get the bad parts: sudo ddrescue -v -r1 /dev/sda /dev/sdb ddrlog.txt (I'm not sure if I need --force with the second command, but I was forced to use it with the first command.) The first command has been running for 45 minutes so far and it reports zero errors and 100 MB/s average throughput. So far, so good. Mike On Sun, 13 Sep 2015, Dan Armbrust wrote: > On 09/12/2015 03:21 PM, Mike Miller wrote: > Wow, Dan, thanks so much for all the ideas!? This is a huge help.? Here's what's going on: > > I already formatted the new drive with ext4 and copied 3 TB of data onto it, so I don't want to undo all that right away, but there is another, probably more appealing way to go: > > /dev/sda and /dev/sdb are identical 2 TB drives that were previously in a RAID1.? It looks like sdb somehow disconnected from the RAID, but sda kept working for a few months before it > failed.? I can mount /dev/sdb1 just fine and I copied all the files off of it onto the new external drive.? I had no errors.? So maybe /dev/sdb is in pretty good shape.? Now that it's backed > up, maybe the best plan is to try to copy /dev/sda to /dev/sdb using one of the dd tools. > > So here's a question:? /dev/sdb is formatted for ext4.? If I want to use it as the destination drive for the dd copy, do I have to use parted to remove the partition table first?? Or what? > > Thanks again! > > Mike > > > You don't have to worry about the partition tables or anything - because rather than copying a partition - such as /dev/sda0 you will just be copying the entire disk - so /dev/sda.? > > When you copy the old (failing) disk onto the other disk (that is either the same size, or larger) you will be copying _everything_ - including the partition table, and the formatting info of the file > system.? The contents of the disk you are writing to will be completely overwritten. > > So then your replacement disk will be (exactly) what the failing disk was, partition table, labels, filesystem and all.? Though, you will likely have some subtle corruption where blocks that couldn't be > read have the wrong bit value.? > > So, if you do manage to recover some things - keep in mind that some of the files my have subtle corruption.? > > Will you notice if 8 bits are flipped in a 2 GB movie?? Probably not... but you would really just have to test the more important files that you recover to make sure they aren't worse than your several > month old backup. > > Dan > > > From mbmiller+l at gmail.com Sun Sep 13 16:02:24 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Sun, 13 Sep 2015 16:02:24 -0500 (CDT) Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: References: <55F4329D.2030408@gmail.com> <55F59381.6050401@gmail.com> Message-ID: At about 400 GB I picked up 4 errors, then nothing for a long time. I left the house for a while and came back to this, which seems bad: $ sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt GNU ddrescue 1.19 About to copy 2000 GBytes from /dev/sda to /dev/sdb. Starting positions: infile = 0 B, outfile = 0 B Copy block size: 128 sectors Initial skip size: 128 sectors Sector size: 512 Bytes Press Ctrl-C to interrupt rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s ipos: 2000 GB, errors: 63, average rate: 49658 kB/s opos: 2000 GB, run time: 5.32 h, successful read: 4.53 m ago Finished It's a 2 TB HDD, so it looks like it did half of it. Any opinions on the best next step? Mike On Sun, 13 Sep 2015, Mike Miller wrote: > Thanks again, Dan. I ended up starting it before getting your message below, > but I think I've got it right. /dev/sdb1 was working and I backed up all of > it. /dev/sda1 couldn't be mounted, but they previously were cloned. So I > unmounted /dev/sdb, installed GNU ddrescue (on a third drive) and did this: > > sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt > > After that finishes, I will run this to try to get the bad parts: > > sudo ddrescue -v -r1 /dev/sda /dev/sdb ddrlog.txt > > (I'm not sure if I need --force with the second command, but I was forced to > use it with the first command.) > > The first command has been running for 45 minutes so far and it reports zero > errors and 100 MB/s average throughput. So far, so good. > > Mike > > > On Sun, 13 Sep 2015, Dan Armbrust wrote: > >> On 09/12/2015 03:21 PM, Mike Miller wrote: >> Wow, Dan, thanks so much for all the ideas!? This is a huge help.? >> Here's what's going on: >> >> I already formatted the new drive with ext4 and copied 3 TB of data >> onto it, so I don't want to undo all that right away, but there is another, >> probably more appealing way to go: >> >> /dev/sda and /dev/sdb are identical 2 TB drives that were previously >> in a RAID1.? It looks like sdb somehow disconnected from the RAID, but sda >> kept working for a few months before it >> failed.? I can mount /dev/sdb1 just fine and I copied all the files >> off of it onto the new external drive.? I had no errors.? So maybe /dev/sdb >> is in pretty good shape.? Now that it's backed >> up, maybe the best plan is to try to copy /dev/sda to /dev/sdb using >> one of the dd tools. >> >> So here's a question:? /dev/sdb is formatted for ext4.? If I want to >> use it as the destination drive for the dd copy, do I have to use parted to >> remove the partition table first?? Or what? >> >> Thanks again! >> >> Mike >> >> >> You don't have to worry about the partition tables or anything - because >> rather than copying a partition - such as /dev/sda0 you will just be >> copying the entire disk - so /dev/sda.? >> >> When you copy the old (failing) disk onto the other disk (that is either >> the same size, or larger) you will be copying _everything_ - including the >> partition table, and the formatting info of the file >> system.? The contents of the disk you are writing to will be completely >> overwritten. >> >> So then your replacement disk will be (exactly) what the failing disk was, >> partition table, labels, filesystem and all.? Though, you will likely have >> some subtle corruption where blocks that couldn't be >> read have the wrong bit value.? >> >> So, if you do manage to recover some things - keep in mind that some of the >> files my have subtle corruption.? >> >> Will you notice if 8 bits are flipped in a 2 GB movie?? Probably not... but >> you would really just have to test the more important files that you >> recover to make sure they aren't worse than your several >> month old backup. >> >> Dan >> >> > From tclug1 at whitleymott.net Mon Sep 14 12:27:21 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Mon, 14 Sep 2015 12:27:21 -0500 Subject: [tclug-list] bluetooth bandwidth Message-ID: do others have experiences, good or bad, with multiple simultaneous bluetooth communications? i wonder if i am hitting a bluetooth bandwidth ceiling with just 2 bluetooth devices (3 counting the dongle), or if it's more likely my CSR 4.0 bluetooth dongle is just crap. i have mplayer playing music on a bluetooth bose, i have linphone setup with a bluetooth headset, a phone call rings in, then the music on the bose gets all wacko crackeley and takes about 25 seconds to recover even after i don't even answer the call. or similarly, while mplayer is playing thru the bluetooth bose, i load a page in firefox which plays a soundtrack into my bluetooth headset, this time the bose is unperturbed, and the headset sounds ok too for a little while but then suddenly goes all wacko crackeley and stays that way. should i hope for software improvements, or try a different bluetooth dongle, or go looking for a wifi headset(do they exist, do they work with linux, do they work with phones, are they good quality)? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykuksenko at gmail.com Mon Sep 14 15:51:49 2015 From: ykuksenko at gmail.com (Yevgeniy Kuksenko) Date: Mon, 14 Sep 2015 20:51:49 +0000 Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: References: <55F4329D.2030408@gmail.com> <55F59381.6050401@gmail.com> Message-ID: Hi Mike, There is a retrim flag to clear the bad sectors from the existing log. Though this will clear all the bad sectors from the log which is not desirable. The single errors are usually truly errors and the goal is to trim the giant blob of failed sectors to some thing more manageable anyway. You may have to edit the log manually. It is pretty simple, the documentation on their website describes the format. If I remember correctly the format is position, size, status (numbers are in HEX). Find the manual here . Section 6 is the logfile (now known as the map file). You can use ddrescueview to see where the bad blocks are distributed on the drive for a more visual idea of what's going on. If it locked at 1 error then just jumped to finish the rest --max-errors=+X may be useful it will stop trying to read the drive after X error(s). I had a drive a while back where it would stop reading the drive completely after the first error and only power cycle would bring it back. That one took nearly 100 attempts to read all the way but I got everything off of it. You should also try the read in reverse flag (--reverse) if it just gets stuck in the center of the drive. Using the direct io flag (on v1.20 of ddrescue --idirect) can help deal with bad sectors as the kernel doesn't handle them (mostly to avoid buffers). I almost always use it when cloning bad drives - it seems to speeds things up and prevents unnecessary retries in reading the media (in theory - I don't know this for sure) On a side note: I usually clone to file if I can - there is a bit more flexibility with manipulating it though a drive should work well too. (I usually end up doing NTFS recoveries from bad drives over the network with raw images mounted after ddrescue saves all the data) Best of luck with this. On Sun, Sep 13, 2015 at 4:02 PM Mike Miller wrote: > At about 400 GB I picked up 4 errors, then nothing for a long time. I > left the house for a while and came back to this, which seems bad: > > $ sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt > GNU ddrescue 1.19 > About to copy 2000 GBytes from /dev/sda to /dev/sdb. > Starting positions: infile = 0 B, outfile = 0 B > Copy block size: 128 sectors Initial skip size: 128 sectors > Sector size: 512 Bytes > > Press Ctrl-C to interrupt > rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s > ipos: 2000 GB, errors: 63, average rate: 49658 kB/s > opos: 2000 GB, run time: 5.32 h, successful read: 4.53 m ago > Finished > > It's a 2 TB HDD, so it looks like it did half of it. > > Any opinions on the best next step? > > Mike > > > On Sun, 13 Sep 2015, Mike Miller wrote: > > > Thanks again, Dan. I ended up starting it before getting your message > below, > > but I think I've got it right. /dev/sdb1 was working and I backed up > all of > > it. /dev/sda1 couldn't be mounted, but they previously were cloned. So > I > > unmounted /dev/sdb, installed GNU ddrescue (on a third drive) and did > this: > > > > sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt > > > > After that finishes, I will run this to try to get the bad parts: > > > > sudo ddrescue -v -r1 /dev/sda /dev/sdb ddrlog.txt > > > > (I'm not sure if I need --force with the second command, but I was > forced to > > use it with the first command.) > > > > The first command has been running for 45 minutes so far and it reports > zero > > errors and 100 MB/s average throughput. So far, so good. > > > > Mike > > > > > > On Sun, 13 Sep 2015, Dan Armbrust wrote: > > > >> On 09/12/2015 03:21 PM, Mike Miller wrote: > >> Wow, Dan, thanks so much for all the ideas! This is a huge help. > >> Here's what's going on: > >> > >> I already formatted the new drive with ext4 and copied 3 TB of > data > >> onto it, so I don't want to undo all that right away, but there is > another, > >> probably more appealing way to go: > >> > >> /dev/sda and /dev/sdb are identical 2 TB drives that were > previously > >> in a RAID1. It looks like sdb somehow disconnected from the RAID, but > sda > >> kept working for a few months before it > >> failed. I can mount /dev/sdb1 just fine and I copied all the > files > >> off of it onto the new external drive. I had no errors. So maybe > /dev/sdb > >> is in pretty good shape. Now that it's backed > >> up, maybe the best plan is to try to copy /dev/sda to /dev/sdb > using > >> one of the dd tools. > >> > >> So here's a question: /dev/sdb is formatted for ext4. If I want > to > >> use it as the destination drive for the dd copy, do I have to use > parted to > >> remove the partition table first? Or what? > >> > >> Thanks again! > >> > >> Mike > >> > >> > >> You don't have to worry about the partition tables or anything - because > >> rather than copying a partition - such as /dev/sda0 you will just be > >> copying the entire disk - so /dev/sda. > >> > >> When you copy the old (failing) disk onto the other disk (that is either > >> the same size, or larger) you will be copying _everything_ - including > the > >> partition table, and the formatting info of the file > >> system. The contents of the disk you are writing to will be completely > >> overwritten. > >> > >> So then your replacement disk will be (exactly) what the failing disk > was, > >> partition table, labels, filesystem and all. Though, you will likely > have > >> some subtle corruption where blocks that couldn't be > >> read have the wrong bit value. > >> > >> So, if you do manage to recover some things - keep in mind that some of > the > >> files my have subtle corruption. > >> > >> Will you notice if 8 bits are flipped in a 2 GB movie? Probably not... > but > >> you would really just have to test the more important files that you > >> recover to make sure they aren't worse than your several > >> month old backup. > >> > >> Dan > >> > >> > >_______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.armbrust.list at gmail.com Wed Sep 16 12:30:45 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Wed, 16 Sep 2015 12:30:45 -0500 Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: References: <55F4329D.2030408@gmail.com> <55F59381.6050401@gmail.com> Message-ID: <55F9A745.1030701@gmail.com> On 09/13/2015 04:02 PM, Mike Miller wrote: > At about 400 GB I picked up 4 errors, then nothing for a long time. I left the house > for a while and came back to this, which seems bad: > > $ sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt > GNU ddrescue 1.19 > About to copy 2000 GBytes from /dev/sda to /dev/sdb. > Starting positions: infile = 0 B, outfile = 0 B > Copy block size: 128 sectors Initial skip size: 128 sectors > Sector size: 512 Bytes > > Press Ctrl-C to interrupt > rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s > ipos: 2000 GB, errors: 63, average rate: 49658 kB/s > opos: 2000 GB, run time: 5.32 h, successful read: 4.53 m ago > Finished > > It's a 2 TB HDD, so it looks like it did half of it. > > Any opinions on the best next step? > > Mike > This is the point where I just end up using google myself... I use the rescue tools infrequently enough that I forget all of the tweaks and things to try with working around failing drives. You might have hit a default limit on the number of errors it will read before giving up... or, as Yevgeniy suggested, try running in reverse. On the plus side, at least your drive is still coming up to the OS. I've had a few, where, the only way I could get it to even appear as a drive was to put it in the freezer... then it would work for about 20 minutes, till it warmed up, then fail again... You just keep trying to get as many blocks read as possible, and then at some point, you call it good - and move on to trying to do the ext4 recovery steps on the restored drive, to try to restore the file system to a state where it will mount. Good luck, Dan From mbmiller+l at gmail.com Wed Sep 16 15:38:45 2015 From: mbmiller+l at gmail.com (Mike Miller) Date: Wed, 16 Sep 2015 15:38:45 -0500 (CDT) Subject: [tclug-list] recovering data from failing ext4 drive In-Reply-To: <55F9A745.1030701@gmail.com> References: <55F4329D.2030408@gmail.com> <55F59381.6050401@gmail.com> <55F9A745.1030701@gmail.com> Message-ID: Thanks again to Dan and Yevgeniy for the info. I tried first doing reverse read, but that just quit immediately: $ sudo ddrescue -v -R -n --force /dev/sda /dev/sdb ddrlog.txt GNU ddrescue 1.19 About to copy 2000 GBytes from /dev/sda to /dev/sdb. Starting positions: infile = 0 B, outfile = 0 B Copy block size: 128 sectors Initial skip size: 128 sectors Sector size: 512 Bytes Press Ctrl-C to interrupt Initial status (read from logfile) rescued: 951404 MB, errsize: 1048 GB, errors: 63 Current status rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s ipos: 0 B, errors: 63, average rate: 0 B/s opos: 0 B, run time: 0 s, successful read: 0 s ago Finished Note that it ran for 0 seconds. So I tried using -r1 to tell it to try to recover more data. That ran for about 16 hours, but it said it didn't recover anything. It did a lot of "scraping". I don't know if it accompliched anything. Here's the output: $ sudo ddrescue -v -r1 --force /dev/sda /dev/sdb ddrlog.txt GNU ddrescue 1.19 About to copy 2000 GBytes from /dev/sda to /dev/sdb. Starting positions: infile = 0 B, outfile = 0 B Copy block size: 128 sectors Initial skip size: 128 sectors Sector size: 512 Bytes Press Ctrl-C to interrupt Initial status (read from logfile) rescued: 951404 MB, errsize: 1048 GB, errors: 63 Current status rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s ipos: 2000 GB, errors: 63, average rate: 0 B/s opos: 2000 GB, run time: 16.71 h, successful read: 16.71 h ago Finished On the bright side, I mounted /dev/sdb1 -- the main partition on the drive I was writing to, and it looked really good. I mounted it read-only. I can find files that I had lost and many of them look perfect. The home movies that I thought were there seem not to be there. So I think I copied them to an external drive, which is good news. I can see that some files are corrupted -- like text files that are now just a mess of binary characters -- but most files I've looked at seem perfect. So I'm pretty happy with the results. Thanks for everything! Mike On Wed, 16 Sep 2015, Dan Armbrust wrote: > On 09/13/2015 04:02 PM, Mike Miller wrote: >> At about 400 GB I picked up 4 errors, then nothing for a long time. I left >> the house for a while and came back to this, which seems bad: >> >> $ sudo ddrescue -v -n --force /dev/sda /dev/sdb ddrlog.txt >> GNU ddrescue 1.19 >> About to copy 2000 GBytes from /dev/sda to /dev/sdb. >> Starting positions: infile = 0 B, outfile = 0 B >> Copy block size: 128 sectors Initial skip size: 128 sectors >> Sector size: 512 Bytes >> >> Press Ctrl-C to interrupt >> rescued: 951404 MB, errsize: 1048 GB, current rate: 0 B/s >> ipos: 2000 GB, errors: 63, average rate: 49658 kB/s >> opos: 2000 GB, run time: 5.32 h, successful read: 4.53 m ago >> Finished >> >> It's a 2 TB HDD, so it looks like it did half of it. >> >> Any opinions on the best next step? >> >> Mike >> > This is the point where I just end up using google myself... I use the rescue > tools infrequently enough that I forget all of the tweaks and things to try > with working around failing drives. > > You might have hit a default limit on the number of errors it will read > before giving up... or, as Yevgeniy suggested, try running in reverse. > > On the plus side, at least your drive is still coming up to the OS. I've had > a few, where, the only way I could get it to even appear as a drive was to > put it in the freezer... then it would work for about 20 minutes, till it > warmed up, then fail again... > > You just keep trying to get as many blocks read as possible, and then at some > point, you call it good - and move on to trying to do the ext4 recovery steps > on the restored drive, to try to restore the file system to a state where it > will mount. > > Good luck, > > Dan > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > From dkoch at link-us.net Wed Sep 16 16:50:03 2015 From: dkoch at link-us.net (Doug Koch) Date: Wed, 16 Sep 2015 21:50:03 +0000 Subject: [tclug-list] Linux Systems Administrator position Message-ID: <74df15299b1b4cd4b8d886836f4a4c8f@exch1.jobsmn.com> I am currently working on a fulltime Linux Systems Administrator position that is in the south metro of the twin cities. Design of system administration-related solutions for various project and operational needs Build servers and configure hardware, peripherals, services, settings, directories, storage, etc. in accordance with standards and project/operational requirements Install, configure, and upgrade systems applications Research and recommend innovative and, where possible, automated approaches for system administration tasks while identifying approaches that leverage our resources and provide economies of scale Perform daily and long term system monitoring, verifying the integrity and availability of all hardware, server resources, systems and key processes, reviewing system and application logs, and verifying completion of scheduled jobs Perform regular security monitoring 5+ years of prior Linux systems administration experience required Experience working with virtual environments such as VMWare or OVM Experience managing SANs a plus If you have an interest in learning more about this, contact me at 952-345-7106 or dkoch at link-us.net Doug Koch Technology Recruiting Manager Link Executive Search -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.krull at gmail.com Wed Sep 16 21:26:10 2015 From: andrew.krull at gmail.com (Andrew Krull) Date: Wed, 16 Sep 2015 21:26:10 -0500 Subject: [tclug-list] Linux Systems Administrator position In-Reply-To: <74df15299b1b4cd4b8d886836f4a4c8f@exch1.jobsmn.com> References: <74df15299b1b4cd4b8d886836f4a4c8f@exch1.jobsmn.com> Message-ID: Doug, Thank you for reaching out. I was wondering with what company is this position for? Thank you again. -- Andrew Krull On Wed, Sep 16, 2015 at 4:50 PM, Doug Koch wrote: > I am currently working on a fulltime Linux Systems Administrator position > that is in the south metro of the twin cities. > > > > Design of system administration-related solutions for various project and > operational needs > > Build servers and configure hardware, peripherals, services, settings, > directories, storage, etc. in accordance with standards and > project/operational requirements > > Install, configure, and upgrade systems applications > > Research and recommend innovative and, where possible, automated > approaches for system administration tasks while identifying approaches > that leverage our resources and provide economies of scale > > Perform daily and long term system monitoring, verifying the integrity and > availability of all hardware, server resources, systems and key processes, > reviewing system and application logs, and verifying completion of > scheduled jobs > > Perform regular security monitoring > > 5+ years of prior Linux systems administration experience required > > Experience working with virtual environments such as VMWare or OVM > > Experience managing SANs a plus > > > > If you have an interest in learning more about this, contact me at > 952-345-7106 or dkoch at link-us.net > > > > Doug Koch > > Technology Recruiting Manager > > Link Executive Search > > > > > > > > > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From woodbrian77 at gmail.com Sun Sep 20 18:16:17 2015 From: woodbrian77 at gmail.com (Brian Wood) Date: Sun, 20 Sep 2015 18:16:17 -0500 Subject: [tclug-list] Got it working Message-ID: On Aug 25, 2015, at 2:54 PM, Chuck Cole wrote: >> I cleaned the old paste off of the chip and applied some new >> paste and everything seems to still be fine. I wasn't sure how >> long to leave the box on it's side while the paste dried. I only >> gave it a few minutes before putting it back up on it's feet. >> > > The paste should not dry. If so it is bad. I see. Perhaps "compound" is a better term for it. I've been using the server now for my site and it's been working well. Currently I have one drive and the operating system and user data are on that drive. I'd like to take another step in this project by adding a second drive for the OS. One question is how big of a drive to buy for that? I was thinking 60gb would be fine. Here are a few smaller enterprise drives: http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&Description=enterprise+ssd&N=100011691%20600546214%20600415805%20600439446&isNodeId=1 One review of the Kingston says it shouldn't be called an enterprise drive. The performance of the Intel drive isn't the greatest. The one that's 8gb and costs $2400 is beyond my budget. Do you have any suggestions for this? Tia. -- Brian Ebenezer Enterprises - So far G-d has helped us. http://webEbenezer.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.armbrust.list at gmail.com Wed Sep 23 14:36:14 2015 From: daniel.armbrust.list at gmail.com (Dan Armbrust) Date: Wed, 23 Sep 2015 14:36:14 -0500 Subject: [tclug-list] Got it working In-Reply-To: References: Message-ID: <5602FF2E.6050208@gmail.com> An HTML attachment was scrubbed... URL: From tclug1 at whitleymott.net Wed Sep 23 20:16:44 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Wed, 23 Sep 2015 20:16:44 -0500 Subject: [tclug-list] bash http Message-ID: this is totally a headscratcher for me. my browser fetches and displays the expected and reasonable pages for google.com, www.kfai.org, and yermande.org. and the first command below seems to successfully fetch google.com's root page. however the second and third commands below get surprising results from www.kfai.org and yermande.org. any ideas what's happening? >$ exec 3<> /dev/tcp/google.com/80;echo "GET /">&3;while read -r r;do echo $r;done<&3|head -5 >HTTP/1.0 200 OK >Date: Thu, 24 Sep 2015 00:17:43 GMT >Expires: -1 >Cache-Control: private, max-age=0 >Content-Type: text/html; charset=ISO-8859-1 >$ exec 3<>/dev/tcp/www.kfai.org/80;echo "GET /">&3;while read -r r;do echo $r;done<&3 >

It works!

>

This is the default web page for this server.

>

The web server software is running but no content has been added, yet.

> >$ exec 3<>/dev/tcp/yermande.org/80;echo "GET /">&3;while read -r r;do echo $r;done<&3 > > >404 Not Found > >

Not Found

>

The requested URL / was not found on this server.

>
>
Apache Server at 127.0.0.1 Port 80
> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tclug at jfoo.org Wed Sep 23 20:47:23 2015 From: tclug at jfoo.org (John Gateley) Date: Wed, 23 Sep 2015 20:47:23 -0500 Subject: [tclug-list] bash http In-Reply-To: References: Message-ID: <5603562B.5090401@jfoo.org> On 9/23/15 8:16 PM, gregrwm wrote: > this is totally a headscratcher for me. my browser fetches and > displays the expected and reasonable pages for google.com > , www.kfai.org , and > yermande.org . and the first command below seems > to successfully fetch google.com 's root page. > however the second and third commands below get surprising results > from www.kfai.org and yermande.org > . any ideas what's happening? > > >$ exec 3<> /dev/tcp/google.com/80;echo > "GET /">&3;while read -r r;do echo $r;done<&3|head -5 ... Why not use curl or wget? If you want to use bash this way, try wireshark/tcpdump to see exactly what's going down the wire. John -------------- next part -------------- An HTML attachment was scrubbed... URL: From o1bigtenor at gmail.com Wed Sep 23 22:22:24 2015 From: o1bigtenor at gmail.com (o1bigtenor) Date: Wed, 23 Sep 2015 22:22:24 -0500 Subject: [tclug-list] Got it working In-Reply-To: <5602FF2E.6050208@gmail.com> References: <5602FF2E.6050208@gmail.com> Message-ID: On Wed, Sep 23, 2015 at 2:36 PM, Dan Armbrust wrote: > "enterprise" is a nearly meaningless buzzword, especially in the SSD drive > space. > > Not SSD specific, but interesting info: > https://www.backblaze.com/blog/enterprise-drive-reliability/ > > https://www.backblaze.com/blog/hard-drive-reliability-stats-for-q2-2015/ > > With rotational drives, some of them that are labeled "enterprise" have > different performance characteristics that are better suited to sitting in a > datacenter, feeding NAS 24x7. > > As far as space for OS only... how big is your OS install? I use partitions > as small as 10 GB for a basic webserver install... > > If you want reliability, buy a pair of 32 or 64 GB drives, put them in a > software raid, and call it done. Greetings That is what I tried to do. Had a raid 10 array for all my data. One drive purely for operating system (with VMs). Well - - - somehow I landed up with a corrupted superblock on one drive on the array. Have had a mountain of hurt trying to get the array read. Raid is not enough - - - I would suggest a raid array (as above) but then backed up to a NAS box with at least daily backups.!! Dee From jay at slushpupie.com Wed Sep 23 23:20:08 2015 From: jay at slushpupie.com (Jay Kline) Date: Wed, 23 Sep 2015 23:20:08 -0500 Subject: [tclug-list] bash http In-Reply-To: References: Message-ID: The problem is you are doing HTTP/1.0, not HTTP/1.1, and very few web servers today do 1.0. Try this: exec 3<> /dev/tcp/www.kfai.org/80; echo -e "GET / HTTP/1.1\nHost: www.kfai.org\n\n">&3; while read -r r; do echo $r; done <&3 -- Jay Kline jay at slushpupie.com On Wed, Sep 23, 2015 at 8:16 PM, gregrwm wrote: > this is totally a headscratcher for me. my browser fetches and displays the > expected and reasonable pages for google.com, www.kfai.org, and > yermande.org. and the first command below seems to successfully fetch > google.com's root page. however the second and third commands below get > surprising results from www.kfai.org and yermande.org. any ideas what's > happening? > >>$ exec 3<> /dev/tcp/google.com/80;echo "GET /">&3;while read -r r;do echo >> $r;done<&3|head -5 >>HTTP/1.0 200 OK >>Date: Thu, 24 Sep 2015 00:17:43 GMT >>Expires: -1 >>Cache-Control: private, max-age=0 >>Content-Type: text/html; charset=ISO-8859-1 > >>$ exec 3<>/dev/tcp/www.kfai.org/80;echo "GET /">&3;while read -r r;do echo >> $r;done<&3 >>

It works!

>>

This is the default web page for this server.

>>

The web server software is running but no content has been added, >> yet.

>> > >>$ exec 3<>/dev/tcp/yermande.org/80;echo "GET /">&3;while read -r r;do echo >> $r;done<&3 >> >> >>404 Not Found >> >>

Not Found

>>

The requested URL / was not found on this server.

>>
>>
Apache Server at 127.0.0.1 Port 80
>> > > _______________________________________________ > TCLUG Mailing List - Minneapolis/St. Paul, Minnesota > tclug-list at mn-linux.org > http://mailman.mn-linux.org/mailman/listinfo/tclug-list > From tclug1 at whitleymott.net Thu Sep 24 08:46:04 2015 From: tclug1 at whitleymott.net (gregrwm) Date: Thu, 24 Sep 2015 08:46:04 -0500 Subject: [tclug-list] bash http Message-ID: >$ exec 3<>/dev/tcp/kfai.org/80;echo -e "GET /CenturySong HTTP/1.1\nHost: kfai.org\n">&3;while read -r r;do echo $r;done<&3 bingo, thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mr.chew.baka at gmail.com Thu Sep 24 08:57:31 2015 From: mr.chew.baka at gmail.com (B-o-B De Mars) Date: Thu, 24 Sep 2015 08:57:31 -0500 Subject: [tclug-list] Got it working In-Reply-To: <5602FF2E.6050208@gmail.com> References: <5602FF2E.6050208@gmail.com> Message-ID: <5604014B.7020404@gmail.com> On 9/23/2015 2:36 PM, Dan Armbrust wrote: > "enterprise" is a nearly meaningless buzzword, especially in the SSD > drive space. > >http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&Description=enterprise+ssd&N=100011691%20600546214%20600415805%20600439446&isNodeId=1 Although it is true that marketing has been diluting the "enterprise" name with cheaper drives based on SOHO, or small office use I do not think one should disregard that there really is a difference between enterprise & non-enterprise SSD's. True enterprise SSD's come with a single level cell (SLC) which will provide longer life. They also come with better controllers, and are sas capable. In my opinion anything not SLC should not be labeled Enterprise. Non enterprise use multi level cells(MLC) (or triple level cells). This allows them to be built cheaper with higher capacity. Down side to MLC or TLC is a shorter lifespan. They also come with cheaper controllers. (like the drive mentioned in the link) My main point here is that there really is a difference between enterprise & non-enterprise. Be careful. There is a valid reason why one ssd is 99.00 while another of the same capacity is 999.00. From mr.chew.baka at gmail.com Fri Sep 25 15:28:38 2015 From: mr.chew.baka at gmail.com (B-o-B De Mars) Date: Fri, 25 Sep 2015 15:28:38 -0500 Subject: [tclug-list] New 500Gig drive for sale Message-ID: <5605AE76.4020803@gmail.com> I have 17 of the following (new) drive: Western Digital - WD5000AAKX - 500Gig - Sata 6gb/s 7200 rpm - 3.5" I also have 4 of the following (new): Seagate Barracuda - ST3500413AS - 500Gig - Sata 6gb/s 7200 rpm - 3.5" $25.00 each These drives were never used. The were shipped in a bunch of Lenovo M92p thinkcenters we ordered. All the drives were pulled upon arrived, and replaced with ssd's. The drives have been sitting on the shelf ever since. If anyone is interested, please email at jedi at grunners.com or call 651-925-1510 We are raising money for charity at work, so all proceeds from the sale of these drives go straight to charity. I also have 50 (give or take) fully buffered RAM chips from HP servers (G6 - G8). Too many to list, so if you need HP Server ram let me know what you need & I will see if I have a match. Thanks! Mr. B-o-B