Tools to Help Document Azure Resource Groups

Sometimes is a good to go back an review any fundamental concepts. This way you can refresh and improve your skill sets. It’s true that practice makes perfect!

So, while reviewing the Microsoft Docs “Azure Architecture Fundamentals” . Here it discuss the use of Dashboards you can customized and organized your Azure resources. This course will help you setup a sandbox environment, so you can build a WordPress web application.

 

I went ahead to create my dashboard  documenting this exercise. Just keeping it simple!

For more information click on Azure Dashboard. You can create up to 100 ‘Private’ dashboards per user.

The next tool has recently got some attention: AzViz PowerShell module. This is module has the ability to generate a visual representation of your Azure Resources.

What was Great about this PowerShell module? For some time, I’ve created many sample demo resource assets within my subscription. This module show me the mess I have with my Azure assets in the cloud. See below image.

This is great tool for documenting your Azure Resources Group with a single one-liner help you create the the an image file of your Azure Resources Group assets.

## - AzViz example, after connecting to your Azure Subscription:
Import-Module AzViz
Export-AzViz -ResourceGroup 'your-azresourcegrp' -Theme light -OutputFormat png -Show -Direction top-to-bottom

Note: Keep in mind, you’ll need to manually rename the output.png file.

As you can see, much better picture presentation compare to the messy one.

This module will can be installed from the PowerShell Gallery, and any issues reported in Github.

As you can see, these tool not only help you to document, but also manage and organize your workloads.

Go Ahead and Try it!

Handling SQL Server Connection strings with PowerShell Secret Management

Finally, I came up with a practical example using the Powershell Secret Management module for storing SQL credentials. This is an excellent way of keeping your SQL connection strings information out of your scripting code. This way we just have it stored in our Vault.

Where do I start?

To install the Powershell Secret Management module, execute the following series of one-liners in a PowerShell prompt to install the latest version from the PowerShell Gallery:

Note: This module has finally reached GA (Generally Available) status.

## - install from the PowerShell Gallery both: SecretManagement, and SecretStore modules:
Install-Module Microsoft.PowerShell.SecretManagement, Microsoft.PowerShell.SecretStore

## - Register the vault with a given name:
Register-SecretVault -Name SecretStore -ModuleName Microsoft.PowerShell.SecretStore -DefaultVault

Now, we got the default “SecretStore” vault created. The vault password will ask once you start adding secrets.

The following cmdlets make it easy to manage your vault:

## - Module: Microsoft.PowerShell.SecretManagement
Get-Secret
Get-SecretInfo
Get-SecretVault
Register-SecretVault
Remove-Secret
Set-Secret
Test-SecretVault
Unregister-SecretVault

## - Microsoft.PowerShell.SecretStore
Get-SecretStoreConfiguration
Reset-SecretStore
Set-SecretStoreConfiguration
Set-SecretStorePassword
Unlock-SecretStore

Note: By-Design. There can only be one vault available.

Take your time to learn these commands.

Let the fun begin

Let’s cut down to the chase and see how this works. I’m going to proceed to create my secret SQL Server connection string values.

Keep in mind, secrets management supports five types of objects: byte[], String, SecureString, PSCredential, and Hashtable. By DEFAULT, the secret will be stored as a ‘SecureString‘ object.

Be creative! Why not store my connection string(s) as a hash table object containing my credential information in the following way:

## - Create hashtable object containing the SQL Connection String:
[hashtable]$MysqlCred01 = @{SqlName = "localhost,1445";Sqlusr = "sa"; SqlPwd = '$MyPwd01!';};

## - This is to veryfy the hashtable object was Properly created:
$MysqlCred01.GetType();
$MysqlCred01

Next after creating the hashtable object, is to save it in the vault with the following command “Set-Secret“:

## - Storing the secret in the vault:
Set-Secret -name MysqlCred01 -secret $MysqlCred01

Note: the first time you store a secret value to the vault, you’ll be prompted for a password.

As you save more secrets, use the following command “Get-SecretInfo” to list what you have in the vault:

## Displaying all stored secrets:
Get-SecretInfo

Now, to get your secret from the vault and use it in PowerShell:

## - Pulling the secret out of the vault into PowerShell variable as plain text:
$MysqlhashCred01 = Get-secret -name MysqlCred01 -asplaintext

## - Accessing hash table values:
$MysqlhashCred01.SqlName
$MysqlhashCred01.Sqlusr
$MysqlhashCred01.SqlPwd

You will notice that eventually, your access will time-out locking you out of the vault. Here’s you use the following command “Unlock-SecretStore” to temporarily unlock the vault:

## - Unlocking the vault to access your secrets providing the vault password:
Unlock-SecretStore -Password '$yourpwd!'

Now, the “Unlock-SecretStore” command is useful for script automation. when you want the script to quickly access the vault. You’ll need to do the following:

## - Unlocking the vault for automation:
Unlock-SecretStore -Password $(ConvertTo-SecureString -String '$yourpwd!' -AsPlainText);
Get-SecretInfo

This way SecretStore vault will not prompt for a password.

Implementing Secret in a GUI Application

Here’s an example of implementing secret in one of my SAPIEN PowerShell Studio GUI applications that check for SQL Server Database Index Fragmentation.

This is a multi-form Window application that where you can select a connection string stored in your SecretStore vault. then you can select the Database and click on the “Start-Job” button to list the status of Database index fragmentation. In this sample application, I can connect to both my local SQL Server and 2 of my Azure SQL Databases.

If you work with PowerShell, both SAPIEN’s Primalscript and PowerShell Studio is a good tool to have for any Administrators and DevOps. Try them out!

For more information

1. Secret Management Blog post.

2. Secret Management in Github. (Post any bugs and/or feedback here)

3. SecretStore in Github. (Post any bugs and/or feedback here)

Have a GREAT SQL PowerShell Day! This is the way!

Getting Ready for PowerShell 7.1 (GA)

This November, PowerShell 7.1 (GA) will become available, as well as PowerShell 7.2 Preview version. And it will come with some interesting features.
If you want more information on these upcoming releases, check out the following two videos:

* Taking your automation to the next level with PowerShell 7

* PowerShell Unplugged – Challenge Edition

Both videos will give you enough information about the history and what’s coming in PowerShell 7.1.
I guarantee that you won’t be disappointed.

But wait! There’s more. Don’t forget to check out any of the existing modules found in the PowerShell Gallery, such as:

* Microsoft.PowerShell.SecretManagement and Microsoft.PowerShell.SecretStore

* Microsoft.PowerShell.GraphicalTools and Microsoft.PowerShell.ConsoleGuiTools

* Microsoft.PowerShell.UnixCompleters

Remember, PowerShell has become the cross-platform automation tool of choice for Windows, Linux, and macOS.
It’s never too late to get on the PowerShell bandwagon!

Installing PowerShell 7 in Ubuntu 20.04

Everyone knows, that sometime soon, Microsoft will support the PowerShell installation in Ubuntu 20.04. But, in the meantime, there’s another way. And, this also applies to WSL (Windows Subsystem for Linux) Ubuntu 20.04.

It is the way!

First, you must follow the instructions for installing .NET Core for Ubuntu 20.04 from the Microsoft Documentation: https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#2004-

Basically, the following commands will install both the .NET Core SDK and the Runtime components:

## Install the SDK

wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb

sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1

## Install Runtime

sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y aspnetcore-runtime-3.1

sudo apt-get install -y dotnet-runtime-3.1

So, after the .NET Core gets installed, then proceed to install the “.NET Global” tool:

## - Install .NET Interactive:
dotnet tool install --global Microsoft.dotnet-interactive

## Install PowerShell Global:
dotnet tool install --global PowerShell

Almost there! There’s one more step we need to do. If you try executing PowerShell, the system can’t find the program.

To resolve the issue of PowerShell not found, we need to add the path to the .NET Global Tools components so that PowerShell can start.

In my case, I open my VIM editor using “sudo” so I can modify the “~/.bashrc” file.

## Add .NET Tools path in Bashrc
$ sudo vim ~/.bashrc
## - Add path to .NET Tools:
export PATH=$PATH:~/.dotnet/tools
:wq

## - Refresh session after updating bashrc:
$ source ~/.bashrc

At this point, now you can start using PowerShell 7 in Ubuntu 20.04.

But, how to update PowerShell?

Simple! The following two commands will update .NET Tools when the update becomes available:

dotnet tool update -g --add-source "https://dotnet.myget.org/F/dotnet-try/api/v3/index.json" Microsoft.dotnet-interactive
dotnet tool update -g powershell

And, the following command will confirm the latest version of both the .NET Tools installed in the system:

dotnet tool list --global

Now, go ahead and have fun with PowerShell.

Happy PowerShelling!!

Available in the Microsoft Store – PowerShell Preview

Yes! If you haven’t noticed by now, PowerShell Preview is available for download from the Microsoft Store.
Just do a search for “PowerShell

It just takes less than a minute to install.

One thing you’ll miss from installing Powershell Preview using the MSI installation! That is, setting the additional options.

After the installation from the Microsoft Store, the PowerShell Preview application settings can be found under the Windows 10 Settings “Apps & features” section.

Then, click on the “Advanced options” to see additional information or make any changes to the application.

Now, the next time there’s an update to the PowerShell Preview, Windows 10 will take care of it automatically.

Happy PowerShelling!

WSL 2 – PowerShell Update-Help cmdlet is not working

Just recently I discovered, when executing the Update-Help cmdlet in WSL 2, that it doesn’t do anything.

Behavior: Run with no progress bar and no error messages at the end of the process. 

I have reported in the PowerShell Github repository and it will be addressed to the proper product group. This is on Windows 10 Version 2004, including Windows 10 Insider edition.

There are two workarounds to this issue:

Workaround #1

The workaround is shown below, thanks to Aditya Patwardhan (Microsoft PowerShell Developer) who provide the hint.

There are two Linux Bash environment variables that need to be updated: LANG and LC_ALL.

Check the current values using the echo command and, in my case, it shows:

## Current values:
(base) maxt@sapien01:~$ echo $LANG
C.UTF-8
(base) maxt@sapien01:~$ echo $LC_ALL
-EMPTY-
(base) maxt@sapien01:~$

Use the following “export” commands to change their values to be “en_US.UTF-8“:

(base) maxt@sapien01:~$
(base) maxt@sapien01:~$ export LC_ALL='en_US.UTF-8'
(base) maxt@sapien01:~$ export LANG='en_US.UTF-8'
(base) maxt@sapien01:~$
(base) maxt@sapien01:~$ echo $LC_ALL
en_US.UTF-8
(base) maxt@sapien01:~$ echo $LANG
en_US.UTF-8
(base) maxt@sapien01:~$

This will fix the issue temporarily during your WSL session, and the Update-Help will work properly.

For now, it may be needed to add these “export …” lines to the “~/.bashrc” file until the fix is available.

Workaround #2

Simply use the “Update-Help” specifying the UIculture:

Update-Help -uiculture en-us

That’s it!!

Keep PowerShelling!

Creating the PowerShell User Profile in Linux

In WSL, as on any Linux distribution, there’s no PowerShell User Profile file(“Microsoft.PowerShell_Profile.ps1“). So, it needs to be created manually.

Creating the profile folder

This profile is stored in the user home configuration folder “~/.config/powershell” folder.

But, the “powershell” folder doesn’t exist, it needs to be created in the configuration folder:

From the bash prompt, follow these steps:

1. Make sure you are in the user home folder:

pwd
cd /home/yourUserFolder

2. Verify the PowerShell folder doesn’t exist:

ls ~/.config

3. Change to the configuration folder:

cd ~/.config

3. Create the “powershell” folder, and assign permissions:

cd ~/.config
mkdir powershell
chmod 755
ll

Creating Microsoft.PowerShell_profile file

1. Using your Linux editor, create the Microsoft.PowerShell_Profile.ps1 file, and add code to the file: (Below using “vim” editor)

sudo vim /home/yourUserFolder/.config/powershell/Microsoft.PowerShell_profile.ps1
-> Write-Host "Welcome to PowerShell in Linux" -foreground 'Yellow';
-> Import-Module Microsoft.PowerShell.UnixCompleters
-> Import-UnixCompleters
-> Write-Host "UnixCompleters is loaded!" -foreground 'Yellow';

5. When done, save changes and exit “vim’ editor by typing:

:wq

Testing the PowerShell Profile

Open PowerShell and the “Welcome to PowerShell in Linux” with any other text will be displayed. At the same time, anything else in the profile will be executed.

Now, you can add more commands to the file when needed.

Keep on PowerShelling!

Streamlining SQL Server Management Objects (SMO) In PowerShell 7 (Revised)

It’s been over two years since I touch this topic, so here’s an updated post about using SQL Server Management Object (SMO) on the latest PowerShell Version 7.

Here’s 411 on what’s out there!

For the most part, nowadays you can use SMO to  connect:

1. Windows to Linux.
2. Linux to Windows.
3. Windows to Linux Containers.
4. Linux to Linux Containers.
5. Windows to Windows Containers.
6. WSL to Linux Containers or Windows.

And, of course, will include cloud technologies.

Now, we have to extend our skills thanks to Docker Container.

*Note: Any connection issues connecting from Linux to Windows, can be solved by creating the inbound rule for Linux in Windows Firewall.

Ways to use SMO

There are two ways you could use SMO in PowerShell 7 (cross-platform):

1. Installing the SMO NuGet packages, two packages are requiered:
a. Microsoft.SqlServer.SqlManagementObjects Version 150.18208.0 (as of 03/23/2020)
b. Microsoft.Data.SqlClient Version 1.1.1 (recommended)

2. Installing the PowerShell Module: SqlServer Version 21.1.18221 (as of 03/23/2020)

Keep in mind, once the packages and/or modules are installed, you need to update them manually.

Working with SMO NuGet Packages

To install the Microsoft.SqlServer.SqlManagementObjects package. You first need to verify that Nuget Package Management is registered in PowerShell 7. Execute the following code will do the task of registration:

function Verify-NugetRegistered
{
[CmdletBinding()]
Param ()
# Microsoft provided code: Test Auto sAVCE
# Register NuGet package source, if needed
# The package source may not be available on some systems (e.g. Linux/Windows)
if (-not (Get-PackageSource | Where-Object{ $_.Name -eq 'Nuget' }))
{
Register-PackageSource -Name Nuget -ProviderName NuGet -Location https://www.nuget.org/api/v2
}
else
{
Write-Host "NuGet Already Exist! No Need to install." -ForegroundColor Yellow;
};
}; Verify-NugetRegistered;

Now, here’s the tricky part. There’s a known issue when executing the Install-Package cmdlet which will fail to install the package.

The workaround is to download the Nuget.exe CLI and place the executable in the following folder: “C:\Program Files\PackageManagement\NuGet\Packages.”

This is the PowerShell default path for storing Packages, and it may not exist in the beginning. So you may need to manually create the folders.

To install the SMO packages needed, execute the following command in PowerShell 7 prompt as an Admin:

cd 'C:\Program Files\PackageManagement\NuGet\Packages\'
./nuget install Microsoft.SqlServer.SqlManagementObjects -version 150.18208.0
Pause
./nuget install Microsoft.Data.SqlClient -version 1.1.1
Pause

Notice, I included the versions of the packages as of 3/23/2020. These SMO packages will support SQL Server 2019 or older, but keeping in mind the older the SQL Server version the latest features will not apply.

Also, these packages doesn’t contain any PowerShell cmdlets, they are meant for developing solution from scratch. For example, below I’m creating an SMO script to connect to a SQL Server providing my SQL authentication, query to get the SQL Server engine version, and manipulate the results from the script.

## - PowerShell 7 loading .NET Core netstandard 2.0 library SMO dll's:
$smopath = Join-Path ((Get-Package Microsoft.SqlServer.SqlManagementObjects).Source `
| Split-Path) (Join-Path lib netstandard2.0);

Add-Type -Path (Join-Path $smopath Microsoft.SqlServer.Smo.dll);
Add-Type -Path (Join-Path $smopath Microsoft.SqlServer.ConnectionInfo.dll);
Add-Type -Path (Join-Path $smopath Microsoft.SqlServer.SmoExtended.dll);
Add-Type -Path (Join-Path $smopath Microsoft.SqlServer.Management.Sdk.Sfc.dll);

## - Prepare login credentials:
$SQLServerInstanceName = 'sapien01,1449';
$SQLUserName = 'sa'; $SqlPwd = '$SqlPwd01!';

## - Prepare connection to SQL Server:
$SQLSrvConn = `
new-object Microsoft.SqlServer.Management.Common.SqlConnectionInfo($SQLServerInstanceName, $SQLUserName, $SqlPwd);
$SQLSrvObj = new-object Microsoft.SqlServer.Management.Smo.Server($SQLSrvConn);

## - Sample T-SQL Queries:
$SqlQuery = 'Select @@Version as FullVersion';

## - Execute T-SQL Query:
[array]$result = $SQLSrvObj.Databases['master'].ExecuteWithResults($SqlQuery);
$GetVersion = $result.tables.Rows;
$GetVersion.FullVersion.Split(' - ')[0];

## - SMO Get SQL Server Info:
$SQLSrvObj.Information `
| Select-Object parent, platform, `
@{ label = 'FullVersion'; Expression = { $GetVersion.FullVersion.Split(' - ')[0]; } }, `
OSVersion, Edition, version, HostPlatform, HostDistribution `
| Format-List;

The best thing! This Package is supported cross-platform so you can execute the script on any OS.

The beauty of coding with SMO is that everything is documented. Just check the Microsoft Documentation “SQL Server Management Objects (SMO) Programming Guide“.

Working with SqlServer Module

Now, using the SQL Server Module in PowerShell 7 is makes it a bit simple to install. And, it’s supported cross-platform.

Just execute the following command as an Admin:

Install-Module -Name SqlServer -AllowClobber

The latest version contains a total of 66 commands you can use to manage your SQL Server engine.

Now, besides having all of these commands available, in the future, you may have the need to create custom functions.

Here’s the variation of the previous SMO script sample:

## - Import the SqlServer module which it loads all needed SMO assemblies:
Import-Module SqlServer

## - Prepare login credentials:
$SQLServerInstanceName = 'sapien01,1449';
$SQLUserName = 'sa'; $SqlPwd = '$SqlPwd01!';

## - Prepare connection to SQL Server:
$SQLSrvConn = `
new-object Microsoft.SqlServer.Management.Common.SqlConnectionInfo($SQLServerInstanceName, $SQLUserName, $SqlPwd);
$SQLSrvObj = new-object Microsoft.SqlServer.Management.Smo.Server($SQLSrvConn);

## - Sample T-SQL Queries:
$SqlQuery = 'Select @@Version as FullVersion';

## - Execute T-SQL Query:
[array]$result = $SQLSrvObj.Databases['master'].ExecuteWithResults($SqlQuery);
$GetVersion = $result.tables.Rows;
$GetVersion.FullVersion.Split(' - ')[0];

## - SMO Get SQL Server Info:
$SQLSrvObj.Information `
| Select-Object parent, platform, `
@{ label = 'FullVersion'; Expression = { $GetVersion.FullVersion.Split(' - ')[0]; } }, `
OSVersion, Edition, version, HostPlatform, HostDistribution `
| Format-List;

The differences is quite simple. All SMO assemblies are previously loaded when you import the SqlServer module. So, you don’t have to worry about including the assemblies in the code. Make sure to check all of the commands available that can help you manage the SQL Server.

Additional Tools Available

Now, don’t forget to check other SQL Server community tools that are available, such as:
1. DBATools – SQL SMO PowerShell.
2. MSSql-Scripter – Python-based tool.
3. Mssql-cli – Python-based tool.

And, don’t forget to check out .NET Interactive which brings Jupyter Notebook with PowerShell kernel.

If you want to try .NET Notebook, I suggest to first install Anaconda (Python 3.7) which makes it simple to use in Windows.

If you want to experiment with .NET Notebook without installing anything in your system, then try MyBinder. This is a web-based .NET Notebook that’s run from a container.

Unfortunately, in this scenario, only the PowerShell 7 core modules are available. But at least you will be able to learn the essentials of .NET Notebook.

Go ahead and start using this Amazing technology!

PowerShell 7 GA is Here!

Finally is here, PowerShell 7 GA (Generally Available) is available for download for Windows, Linux, and macOS. Go and get it!

Installation

I suggest to manually uninstall all previous PowerShell versions and remove all existing folders that will be left behind under the “C:\Program Files\PowerShell” folder. This will guarantee a clean installation of PowerShell 7 GA.

This version will replace any previous GA version of PowerShell. In other words, if you already had PowerShell v6.2.4 installed, it will be replaced with PowerShell v7.0. This is by-designed!

You can find more information about PowerShell 7 GA in the following links:

Update your PowerShell Notebook

Also, check out .NET Interactive/PowerShell Notebook, as it has been updated to support the PowerShell 7 Kernel.

If you have previously installed .NET Interactive, to get the latest PowerShell Kernel, run the following command:

dotnet tool update -g --add-source "https://dotnet.myget.org/F/dotnet-try/api/v3/index.json" Microsoft.dotnet-interactive

For more information here, .NET Interactive/PowerShell Notebook.

Wait, there’s more!

Things are getting better! Check out the preview of the ConsoleGuiTools module for PowerShell 7 but, for now, only available for Linux and macOS.

It’s never too late to learn PowerShell!

Anaconda and PowerShell working together!

Yes! To my surprise, when I completed installing the latest update of Anaconda (Anaconda3 2019.10 (64bit) v4.8.1), I realized they have included the following menu item: “Anaconda PowerShell Prompt (Anaconda3)“. Apparently, this menu item has been added for some time.

So, we can take advantage of this shortcut, especially when we can use this console prompt for working with “PowerShell Notebook.” Please, check out Rob Sewell blog post on the recent update .NET Notebook Preview 2 post about “New .NET Notebooks are here – PowerShell 7 notebooks are here.“.

But, Wait! Let’s take this a little further and get you ready to do some fun.

What’s the main advantage?

The “Anaconda PowerShell Prompt” shortcut is already set to activate Anaconda to be used with Windows PowerShell. There’s no need to do a manual activation by opening a DOS command shell and executing:

c:\> conda activate

Trying to use Python without activating Anaconda, it will give you a message.

The activation will allow you to use Python within Windows PowerShell. Or, just use the shortcut “Anaconda PowerShell Prompt.”

As you probably will notice, this menu item only open Windows PowerShell. So, what about PowerShell Core?

This is probably because of PowerShell Core has multiple versions: PowerShell 6.2.4 (GA) and PowerShell 7 Preview (RC2), both supported by Microsoft.

Would you like to create the Anaconda Pwsh7 Prompt shortcut?

Yes! We can create our own PowerShell Core shortcut. And, here’s how to create the shortcut for Anaconda PowerShell 7 Preview.

First, I will make another copy of the original shortcut and label it “Anaconda Pwsh7-Preview Prompt (Anaconda3)“.

Here’s the original path use the Windows PowerShell shortcut:

%windir%\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -NoExit -Command "& 'C:\ProgramData\Anaconda3\shell\condabin\conda-hook.ps1' ; conda activate 'C:\ProgramData\Anaconda3' "

And, here’s my shortcut modification to use PowerShell 7 Preview:

%ProgramFiles%\PowerShell\7-preview\pwsh.exe -ExecutionPolicy ByPass -NoExit -Command "& 'C:\ProgramData\Anaconda3\shell\condabin\conda-hook.ps1' ; conda activate 'C:\ProgramData\Anaconda3' "

Keep in mind, you will need administrator privileges to create this shortcut in the ProgramData Anaconda menu.

After making all the necessary changes to the new shortcut, we got both Window PowerShell and PowerShell 7 Preview working with Anaconda.

Now go ahead and expand your scripting knowledge!