Powering arrays seperatly in Windows?

[ddrueding]

Thanks to the help of many of you, I now have the data array on my system being powered by a separate PSU. This got me thinking. I only need the array to be available when I am watching movies or listening to music (15% of the time?). So I'm playing with the idea of powering up/down the array as needed.

I think the only thing I need to do is command the 3Ware controller to rescan for drives. At that point it will detect the drives, identify them as being part of an array, and make that array available in Windows.

Does anyone know how to do that without going into the web interface for the controller? I'd love an icon I could double-click.

Any other issues with my evil plan?

~David

 

[Bozo]

This looks interesting:

http://support.microsoft.com/kb/311272

Bozo :joker:

 

[Bozo]

Or posibly running your array from a virtualized OS.

Bozo :joker:

 

[Fushigi]

How will you gracefully power down the array? Obviously for data integrity purposes you need to make sure caches are flushed and there aren't any open files.

Just asking because I've never had the need to do it and wasn't familiar with Windows having a command to do so for storage that it considers internal.

 

[Mercutio]

If that storage was under Linux, you could properly unmount those disk drives and know it's safe to turn them off. The problem with doing this in a VM is that his storage effectively becomes nonlocal, which is the whole point of his configuration anyway.

 

[ddrueding]

How will you gracefully power down the array? Obviously for data integrity purposes you need to make sure caches are flushed and there aren't any open files.

Just asking because I've never had the need to do it and wasn't familiar with Windows having a command to do so for storage that it considers internal.

If that storage was under Linux, you could properly unmount those disk drives and know it's safe to turn them off. The problem with doing this in a VM is that his storage effectively becomes nonlocal, which is the whole point of his configuration anyway.

Good points both. And I'm not sure how I can unmount a drive in Windows.

In order to get the drive to detect after power-up, I need to:

Go to https://localhost:88/
Log in as administrator
Go to the Management Tab and Maintenance Sub-Tab
Press the Rescan Controller button


The easy way for now will be to simply reboot either with the array on or off for a session, but I would like it to be simpler than that (in case the GF wants to watch something while I'm gone).

Thanks for all your help.

 

[Handruin]

Does 3ware offer any type of API or web services?

 

[ddrueding]

Does 3ware offer any type of API or web services?

Good thought. I'll have to take a look.

 

[Handruin]

http://www.3ware.com/KB/article.aspx?id=13446

Maybe you can send them an e-mail to see if they'll send it to you? See if they have any other API's while you're at it. If it can do what you need, I could write you a little app in C# if you needed something specific done outside of their CLI.

 

[Handruin]

Does anyone know how to do that without going into the web interface for the controller? I'd love an icon I could double-click.

Also, if the CLI API can do the rescan, I can make you the little application which resides in the toolbar or even an EXE you can double click to do what you need. Just pick a fun icon that you want to double click and it'll work with it.

If you feel adventurous, you can even try it too using the visual C# express edition which is free to download.

 

[ddrueding]

Thanks for the offer Handruin, I will likely take you up on it when I hear back from them. Their customer service is excellent (as you would expect from a company that makes Enterprise-grade gear), so it shouldn't be long.

 

[Handruin]

Did they ever get back to you with the API?

 

[ddrueding]

The mentioned something about it being on the CD, but I haven't looked yet. Work has been really crazy lately. Sorry.

 

[Handruin]

No big deal. I was just antsy to send you a gui tool. I've spent some time on it and it launches notepad pretty well. ;-) I was having fun playing around with some of this stuff, that was all.

 

[ddrueding]

I really appreciate the thought/effort, and I will get to it soon. For the record, I won't want to test anything until the array finishes rebuilding. 5 days so far and it's at 48%.

 

[Mercutio]

My way of doing things, with the multiple "small" arrays, is starting to make more sense now, isn't it?

 

[Handruin]

No rush, I was just having fun. I can still send you what I have (with source) and you can watch it gloriously open notepad and anyone else can look and see I'm not writing you some mysterious code which deducts $.0001 per day out of your bank account.

At least you can tell me if it will do what you need in terms of user interface and functionality with controlling even something as basic as notepad. Beyond that, it shouldn't be hard to make it use whatever tool 3ware offers. The tool will require the dotnet runtime to be installed.

 

[ddrueding]

My way of doing things, with the multiple "small" arrays, is starting to make more sense now, isn't it?

Meh. One week every few years to add capacity, and the rest of the time I don't need to worry about distributing the data or explaining to the S.O. why the movies are on drive M except for the documentaries because they didn't fit...so they are on drive T with the TV shows...

I will happily call it a wash, but you can't witness the glory of seeing a single drive letter with significant TBs! Muahaha!

I still need one of these (see attachment)

 

[ddrueding]

It seems that they meant the CLI was on the CD (which it is, and the manual for it is here). I've sent a more specific e-mail now, and hope to get a better answer.

 

[Handruin]

I bet the contextual switching with a machine containing 64 CPU's is huge. I'd love to have that for our SQL server at work...

 

[Handruin]

I'll check out the manual and modify the code to suit their API/CLI. If you want to see and play with what I've made so far, check it out here. I made the assumption you might want something that sits in the system tray and is very light weight?

Launcher Binary | Launcher Source

Let me know if this could work for what you might want/need and if not, let me know what to change to fit your needs. Right now it only launches notepad as a test, but I can map it to anything like the 3ware API.

I've included the source in case you or anyone else wants to look at it and make your own build if you don't want to use the compiled binary. This will work with the free Microsoft visual studio express I linked earlier (and it was also built using the free version).

 

[ddrueding]

Sweet, thanks Handy. It even has a right-click menu from the system tray (very cool).

 

[Handruin]

Few questions to go along with the API...? What OS are you running, and is it 32/64 bit? What is the path to the API's CLI once installed...I could also make an external properties file where you could enter this in which might be a fun learning exercise for me.

I looked through the whole doc and didn't see a command to turn off the disks so I'm assuming you'll just power them down manually or some how flush them from the system before powering it down? Dare I suggest it might be possible to tap into the windows disk management, but that's far beyond my expertise and I'd not want to risk your data with it...but I'll see if I can find anything.

There is a rescan feature which is described as such:

/cx rescan [noscan]
This command instructs the controller to rescan all ports and reconstitute all
units. The controller will update its list of ports (attached disks), and attempts
to read every DCB (Disk Configuration Block) in order to re-assemble its
view and awareness of logical units. Any newly found unit(s) or drive(s) will
be listed.
noscan is used to not inform the operating system of the unit discovery. The
default is to inform the operating system.
Example:
//localhost> /c1 rescan
Rescanning controller /c1 for units and drives ...Done
Found following unit(s): [/c1/u3]
Found following drive(s): [/c1/p7, /c1/p8]
9000a No Yes Yes N/A Yes Yes No No No
9650SE No Yes Yes Yes Yes Yes No No No
a. Models 9500S, 9550SX, and 9590SE
Table 8: Supported Model-Unit Types for ignoreECC
Model R-0 R-1 R-5 R-6 R-10 R-50 Single JBOD Spare
Note: If you are adding new drives, add them physically before issuing the rescan
commands. Hot swap carriers are required unless you first power-down the system
to prevent system hangs and electrical damage.

When your array is finished rebuilding and you have the time, can you test the rescan api command and let me know if it does what you need? Also, there are the following two commands that might help you out before powering down the array. I could work these into the GUI tool with a button before you power it down:

/cx commit
This command only applies to the Windows operating system. It commits all
changes if a faster shutdown method is needed when running certain database
applications. Linux and FreeBSD file systems do not require this command
since they have their own ways of notifying the controller to do clean up for
shut down.
/cx flush
This command forces the controller to write all cached data to disk for the
specified controller.

 

[Handruin]

Sweet, thanks Handy. It even has a right-click menu from the system tray (very cool).

No problem, I've been having fun learning. I spent the majority of the time dealing with the thread management...only to find out afterward I should have dug into the thread pooling rather than manage it myself.

I made some assumptions on how you might use the tool, figuring it would run in the task try minimized (though it doesn't really need to). When minimized it was only using about 800K of system memory and about 4MB when restored. Let me know what you might want out of it.

 

[ddrueding]

Unfortunatly, I can't have the API.

David, the API has intellectual property as part of the code so we cannot release it except under NDA to qualified OEM accounts.

But the rescan looks to do exactly what I want, and the commit and flush commands are darn useful as well. It will probably be broken down into a "start" and "stop" sequence with a prompt "Please turn on the array" before a rescan and an "it is now safe to turn off the array" after a commit and flush.

Right now my OS is Vista 64-bit, but it might become Server 2008 64-bit in the future. I'm at 8GB of RAM, so 64-bit-ness is permanent.

Thanks again.

 

[ddrueding]

Oh, and the CLI lives in it's default folder, at C:\Program Files\AMCC\CLI

 

[Mercutio]

Meh.

You can simulate the same effect by using NTFS volume mount points, so that array#1 has 4TB for your music + your documentaries, and array#2 has 4TB for all your TV shows and array#3 has 4TB all your solo female porn and array and array #5 has some of the lesbian porn and array#6 has the rest of the lesbian porn.

Or, instead of volume mount point, under Linux you can mount network shares and then share the directory where you did the mounting.

No, there's not a single, massive space, but you really don't need to spend much time worrying about how the space is distributed, either.

 

[Handruin]

Oh, and the CLI lives in it's default folder, at C:\Program Files\AMCC\CLI

I'm confused, so you can't get the API, but you do have access to the CLI (which is fine if you can only get the CLI)? If so, I'll configure like you requested with the steps for powering on and off. I made a field to enter to location of the CLI so you can install it anywhere and point to it with a browse button.

I'm guessing the tool I have so far worked on vista 64-bit so there's not an issue there? I'll see if there is anything I need to do to make it more 64-bit friendly but I think the common run time library takes care of it for me.

 

[ddrueding]

You got it Handruin. I have the CLI installed at that location, but they won't give me the API. Your application ran fine, so no need to optimize for 64-bit-ness. Everything sounds great so far.

 

[Handruin]

New version if you want to let me know what you think. In the settings tab, just point this to notepad or some other simple application to play. You could pass it the "tw_cli.exe" for your array, but be careful if it's still rebuilding.

Edit: I'm working on adding an XML doc to store setting for the application location path in the settings tab, but for now it's not implemented, so each time you open the tool, you have to re-point it to the ".exe" file. If you forget it will warn you that there is no application path defined.

Version 1.0.0.1 binary

Version 1.0.0.1 source

 

[Handruin]

There is also some bug I introduced when you minimize it to the task tray. It will consume 100% CPU time (it used both my cores)...I'll track it down for the next build.

 

[ddrueding]

It doesn't max my cores on minimize, but it does consume about 12MB of memory. Not an issue (the desktop window manager alone is taking 260MB), and I have plenty, just letting you know in case that wasn't expected.

 

[ddrueding]

Yay! Have some error messages!

Situation:

launcher.exe copied to c:\Program Files (x86)\
Ran the app from that location
Entered the following lines to the "Settings" Tab:

C:\Program Files\AMCC\CLI\tw_cli.exe
/c0 rescan
/c0 commit

Then I pressed the "Prepare to Stop Array" button and instantly received this error.

The error referenced "..//..//..//Stop-Disabled-64x64.png." But it did show the greyscale stop image...

See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.IO.FileNotFoundException: ..//..//..//Stop-Disabled-64x64.png
   at System.Drawing.Image.FromFile(String filename, Boolean useEmbeddedColorManagement)
   at launcher.LauncherForm.StopArrayButton_Click(Object sender, EventArgs e)
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.312 (rtmLHS.050727-3100)
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v2.0.50727/mscorlib.dll
----------------------------------------
launcher
    Assembly Version: 1.0.3133.40225
    Win32 Version: 1.0.0.1
    CodeBase: file:///C:/Program%20Files%20(x86)/launcher.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.312 (rtmLHS.050727-3100)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.312 (rtmLHS.050727-3100)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.312 (rtmLHS.050727-3100)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

It does look like it did what it was supposed to do, but I'm not certain yet.

 

[ddrueding]

Additionally, does anyone have suggestions about what to do to be sure there is no activity to the drive when I power it down? 3Ware suggest removing the drive letter, but it would still be shared, no? How do you unmount a drive in Windows?

 

[ddrueding]

OK. Yes, this programs works with the settings named above. It does still show the error message when pressing either button, it isn't saving it's settings, and I'm still nervous about powering down a mounted drive, but the rest is working perfectly.

Thank you so much!

 

[Handruin]

Additionally, does anyone have suggestions about what to do to be sure there is no activity to the drive when I power it down? 3Ware suggest removing the drive letter, but it would still be shared, no? How do you unmount a drive in Windows?

Does vista 64 have diskpart.exe? If so, this might work and I could script it into the application.

http://www.dslreports.com/forum/r20604782-Easy-way-to-mountunmount-in-windows

I fixed the bug with the images throwing an exception I've simply embedded it into the exe file and it no longer becomes an issue. I've also fixed the high CPU usage due to a threading problem.

Last step is to get this thing to save your user preferences which I'm working on now. If I get it working I'll post the new code tonight.

 

[ddrueding]

Nice find Handruin, it works perfectly.

 

[Handruin]

Can you send me the commands to run if you want it incorporated into this tool? I can call it after the commit/flush, or add it as its own button. Let me know what you'd like if anything. Also, don't feel obligated to use this tool. I'm getting something out of this also (learning) even if you don't use it.

 

[ddrueding]

Strike that. I tried it with a thumb drive, but it doesn't work with GPT disks.

 

[Handruin]

Darn, I was hoping that would work. I did some research about managing windows disk using C# and it doesn't seem likely. I'd have to go to C++/Assembly for that access and I have to admit I'm rusty at C++ and zero skills with assembly.