On 2013/09/04 4:54 PM, "Pablo Escobar" <pescobar001(a)gmail.com> wrote:
could you post or link some more detailed technical docs about how
you
snapshot and backup the mds?
My main doubt is that if I do an snapshot and backup of mds, for example,
on friday and during the weekend the client nodes modify the filesytem
and then I need to recover on monday. wouldn´t it be inconsistent? would
be possible to recover the
data in this situation? Is there any docs about this recover procedure?
Snapshots are done using normal LVM commans "lvcreate -s" to create a
snapshot of the MDT device. I'm sure there are equivalent dmadmin
commands to do the same?
Then, simple "dd" to copy the MDT snapshot to a separate block device
(actually, two separate devices on alternate days, and one only manually).
The MDT is small, and the backup devices do not need to have good IOPS
performance, just simple 5400-RPM SATA is enough for streaming "dd"
copies. Since the MDT is a single point of failure for all of the data in
the filesystem, it makes sense to keep this backed up even if the data is
too large to back up.
This is only intended for emergency recovery, since I also test
experimental patches on my home system at times, but it holds real data
that the rest of my family cares about (music collection, all of the TV we
watch, etc).
Yes, if there is a recovery from the backup MDT copy, the newer files are
lost, but that is true of any backup system. In theory it would be
possible to reconstruct the new files from their OST objects using lfsck
(in my case mostly just newly-recorded TV shows), but that has not been a
primary concern compared to being able to recover the rest of the
filesystem.
Cheers, Andreas
2013/4/10 Dilger, Andreas <andreas.dilger(a)intel.com>
On 2013/09/04 9:47 AM, "Kit Westneat" <kwestneat(a)ddn.com> wrote:
>You should do that in order to get a clean image. If you don't, then you
>are more likely to get an inconsistent image. Of course, an inconsistent
>image is better than no image, but ideally you'd want to get a clean one.
Or, since this is a device mapper device, you can create a snapshot, and
then
use "dd" from the MDS snapshot (which should be consistent). That's what
I
do for my MDS backups, and it has saved me a couple of times.
Cheers, Andreas
>On 04/09/2013 11:33 AM, Kumar, Amit wrote:
>> Hi Kit
>>
>> This is good to know. Do I still need to shutdown the lustre FS? And
>>Umount the devices?
>>
>> Thank you
>> Amit
>>
>> Sent from my iPhone
>>
>> On Apr 9, 2013, at 10:27 AM, "Kit Westneat"
>><kwestneat@ddn.com<mailto:kwestneat@ddn.com>> wrote:
>>
>> Hi Amit,
>>
>> If you are doing a disk to disk backup, and the size of the backup disk
>>is >= the main disk, I would just use dd and not worry about formatting
>>the disk beforehand. The dd will replicate the filesystem structure
>>exactly, so there's no need to mkfs. I would do something like 'dd
>>if=/dev/dm-0 of=/dev/dm-2 bs=1M'.
>>
>> - Kit
>>
>> On 04/09/2013 02:40 AM, Kumar, Amit wrote:
>> I forgot to paste the link I referenced to perform backup: here it is:
>>http://wiki.lustre.org/manual/LustreManual18_HTML/BackupAndRestore.html
>> Any thoughts on this will be very helpful!
>> Thank you,
>> Amit
>>
>> From:
>>hpdd-discuss-bounces@lists.01.org<mailto:hpdd-discuss-bounces@lists.01.o
>>r
>>g> [mailto:hpdd-discuss-bounces@lists.01.org] On Behalf Of Kumar, Amit
>> Sent: Monday, April 08, 2013 12:47 PM
>> To: HPDD-discuss@lists.01.org<mailto:HPDD-discuss@lists.01.org>
>>(HPDD-discuss@ml01.01.org<mailto:HPDD-discuss@ml01.01.org>)
>> Subject: [HPDD-discuss] Lustre Disk to Disk MDS Backup Question
>>
>> Dear All,
>>
>> ³Version 1.8²
>>
>> We have both MGS & MDT mounted on two different block devices.
>> Now I would like to perform a disk to disk backup of both of these
>>block devices to another set of block devices.
>>
>> Although this link tells me how to perform that. There are some missing
>>things that I want to understand before I can perform this backup
>>
>> We have two devices /dev/dm-1 & /dev/dm-0 mounted as /mgs and /mdt
>>respectively.
>>
>> Now I have two more devices /dev/dm-2 & /dev/dm-3 which I want to mount
>>it as /mgs-backup and /mdt-backup and then perform the disk to disk
>>backup. Because we are not backing up any of the metadata right now.
>>
>> Here is where I get lost: when I want to make the lustre file system it
>>asked me if the device is going to be an mgs or mdt or it will be
>>combined. Now that I already have an /mgs and /mdt partition on this
>>node what will happen if I make new block device to be an mgs and mdt
>>partition, will this conflict with the existing ??????
>> Or can I just make the new devices just an ext3 partition and still be
>>able to make a disk to disk copy of meta data that I can restore when
>>needed.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division