Easy pivot: From key-value pairs to columns

This article was recently published on dev.getroadmap.com:

If there is one thing I pretty much hate doing in T-SQL it’s PIVOT and UNPIVOT. Even though I used it a few times in the last couple of years, it’s an adventure to find out how it works every time. And I know a lot of people struggle with this part of T-SQL, so let’s take a look at a (hopefully) simple example.

Key-Value pairs
The idea of using key-value pairs to store data isn’t new, but with the rapid development of cloud solutions like Azure and the hype around NoSQL databases, using key-value pairs to store data got a big boost. Especially developers (in my experience) love using key-value pair to store their data, because it’s easy for them to consume the data in an application. But it gives the database professional an extra challenge because we’re used to retrieve columns with values instead of a record per value. So how can we turn those key-value pairs into rows?

To show you how to change key-value pairs to columns, let’s create an example based on flights. If one of our customers needs to travel and they take the plane, there is some basic information we need to show them the flight status in the app. So let’s create a table to store that dataset:

CREATE TABLE dbo.Flights
	[Key] VARCHAR(255),
	[Value] VARCHAR(255)

and insert a flight:

INSERT INTO dbo.Flights
	(FlightId, [Key], [Value])
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'FlightId', '3074e015-62b2-4f76-a8b1-463c53cd79c5'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'AirlineIATACode', 'VY'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'FlightNumber', '8336'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'DepartureAirportCode', 'RTM'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'DepartureAirportName', 'Rotterdam The Hague Airport'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'DepartureDateTime', '2016-08-12 12:15'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'DepartureTerminal', '1'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'Gate', NULL),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'ArrivalAirportCode', 'BCN'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'ArrivalAirportName', 'Barcelona–El Prat'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'ScheduledArrivalDateTime', '2016-08-12 14:14'),
	('2A0C8B05-682A-41CE-8516-C6070CD92851', 'FlightIsCancelled', '0')

So now we have a dataset that consists of 12 key-value pairs:

Converting to columns
But when you’re running a query you would like to have columns instead of rows. So by using a PIVOT statement, you can convert these rows into columns:

		FROM dbo.Flights
	) AS SourceTable
	MIN([Value])	--Needs to be an aggregate function
	FOR [Key] IN
	) AS PivotOutput

Because the keys are always the same for flights, I can add those Key-names in the PIVOT statement. But because the PIVOT statement needs an aggregate function to retrieve the value, I needed to add the MIN() function but this doesn’t change the output in any way.

Now we have the same dataset, but converted into rows:

So from now on, converting key-value pairs to columns shouldn’t be a problem anymore!

Tracking query progress with Live Query Statistics

How frustrating is it, to run a query on a database and it seems to be “stuck”. We’ve all seen that happen right? But how can you tell what the problem is, without letting the query complete (which could take a long time)? Microsoft (or actually the SQL Server team) gave us a new toy to play with that can help us in situations like this, and this tool is called “Live Query Statistics”.

Creating resources
To show you how the Live Query Statistics work, let’s create a sample table first, and insert 1.000 rows:

INSERT INTO LiveQueryStats
GO 1000

Because we want to actually see something happen, let’s multiply the number of rows coming from the single table by using the query below:

FROM LiveQueryStats T1
CROSS APPLY LiveQueryStats T2
CROSS APPLY LiveQueryStats T3

The query will return 1 billion rows (, and will run for a while. This gives you the opportunity to look at the different features without rushing or losing your running query.

Live execution plan
Before running the query above, you need to enable the Live Query Statistics just like you would do to the normal execution plan. This is an extra button added in the SQL Server 2016 SSMS (and the downloadable version of course). When that’s enabled, you can execute the query, and SQL Server Management Studio (SSMS) will automatically switch to the live execution plan.

In this execution plan, you’ll see the data flow through the components, so you can actually see what SQL Server is doing at a specific moment:

Query completion percentage
Another great addition is the overall completed percentage, that you can find at the bottom of your SSMS. This shows you the percentage of completion, that can help you estimate the time till completion:

Live rowcount
The properties window normally shows you the in-depth information of your query after completion (number of threads, memory grant, etc). With the live execution plan enabled, it can provide you with real-time statistics as well:

Live execution plan from Activity Monitor
From the Activity Monitor you can also open the live execution plan of running queries on your instance. In the Activity Monitor, you have an additional tab in SQL Server 2016 called “Active Expensive Queries”. In this tab you can right-click on a running query, and click on “Show Live Execution Plan”. This opens a new tab in SSMS with the execution plan:

Unfortunately this only works for queries that have the live statistics enabled before execution. I don’t expect this to change in the final product, mainly because of the negative performance impact this feature can have on your queries and instance.

Drawbacks, Limitations and Bugs
Because this is only a CTP version of SQL Server 2016, we can expect some bugs and limitation, so I’ll be the last one to judge. And I don’t think these limitations are a big drawback on the feature.

One of these bugs is a crashing SSMS. If you open a live execution plan from the Activity Monitor, and close that tab, SSMS crashes every now and then.

Another weird thing is that exiting SSMS by clicking on the close button (top right) when the live execution plan is open, causes the list of recently used SQL Servers (the list used when connecting object explorer for example) to be cleared for some reason.

This behavior is reproducible, so I think this is a small bug in SSMS, or the fact that I upgraded this instance from the first CTP version till the current version. But I’m sure this will be fixed by the SQL Server Team in the next releases.

The biggest pitfall (in my opinion) of this feature is also mentioned in the documentation. This feature is “primarily intended for troubleshooting purposes” and “can slow the overall query performance”. And I know, it look SO COOL to have this on all queries you’re running, but please be careful with this. Don’t enable this on every running query, but only use this to debug issues!

Another limitation for this feature (at least at the moment I’m writing this), is the use in combination with columnstore indexes, memory optimized tables and natively compiled stored procedures. You can read more about this here.

And I shouldn’t even have to mention this, but remember: you can only use Live Execution Plans when you have SHOWPLAN permissions on the database (same permissions you need to view normal execution plans).

One of the questions I had when I read about this: are there any alternative for this? One of the only things that come to mind is the “Track My Query” tool, written by Matan Yungman (Blog | @MatanYungman). This tool allows you to monitor your query, and it will show you what part of your query is currently running. For more in-depth information, I recommend the SQLBits session where he explains the inner-workings of this tool. I’ve seen this session in person, and it was an interesting session.

With all these awesome new features that will be shipped in SQL Server 2016, I think there’s a whole new way of looking at SQL Server as a product. I’ve always liked working with SQL Server, but this version is taking it to the next level.

With all these new features, debugging issues is going to be a bit easier, developing new stuff is going to be faster, and the overall usability is going to skyrocket. I can’t wait to get my hands on the finished product!

Searching through the SQL Server error logs

SQL Server has a number of error logs, where both informational messages and errors are logged. You can compare it to the event viewer in Windows, but than only for SQL Server. This error log contains a lot of potentially useful information when you’re investigating an issue.

The physical location of the logfiles is “C:\Program Files\Microsoft SQL Server\MSSQL12.[InstanceName]\MSSQL\Log”. This can also be different if you changed the path in the SQL Server setup (for example, the path on my machine is: “C:\Program Files\Microsoft SQL Server 2014\MSSQL12.[InstanceName]\MSSQL\Log”). In that directory you’ll find a number of ERRORLOG.[Number] files. There is a file for every archive, which depends on your SQL Server configuration. You can open the files with notepad, or any other text-editor you like.

But you can also access these archives from SQL Server Management Studio (SSMS).

SQL Server Logs
If you connect the object explorer in your SSMS, you can navigate to Management -> SQL Server Logs. There you see the number of configured log files (default is 7 log files: 6 archives + current log file):

You can double-click a log file to open it. In the pop-up window you see the contents of the log, a number of checkboxes on the left to add more archives to the current view, and a button called “Filter…” that you can use to filter the current view:

But unfortunately the filter in the Log File Viewer isn’t always as easy to use. For example, you can’t (at least as far as I know) filter on 2 strings. So how are you going to look for any events that contains “Backup” or “Restore”? That’s not possible with this filter.

If you would rather use T-SQL to find things in the SQL Server Error Log, that’s also possible. There’s an extended procedure called xp_readerrorlog you can use for that, or you can use sp_readerrorlog (which is a stored procedure that used xp_readerrorlog).

If you use sp_readerrorlog or xp_readerrorlog, you can also apply filters. You can use a number of parameters to filter the output, but you can only do so on 1 singe log file:

EXEC xp_readerrorlog
	0,						--ArchiveID (First error log = 0)
	1,						--Log type (SQL Server = 1, SQL Agent = 2)
	N'Backup',				--Filter ProcessInfo
	N'Sandbox',				--Filter Text
	'20150826 00:00:00',	--DateFrom
	'20150828 10:32:00',	--DateTill
	'ASC'					--Sort order

By altering the filter-parameters (or leave them empty), you can search through the SQL Server error log or the SQL Server Agent error log, and you can do so on a number of columns and variables (like datetimes). But you still have the same issue you had with the log viewer in SSMS: you can can only search on 1 filter, and you can’t search through more than 1 archive at the same time. So how can we solve that?

Search trough multiple archives
Searching through multiple log files might come in handy if you’re not sure if the database was taken offline today or last week. Or if you’re not sure if the problem occurred before or after a log file cycle.

One way of doing this (which I use on a regular basis, and I blogged about earlier), is by inserting all the log information in a temporary table and search through that results:

	 ProcessInfo VARCHAR(100),

/* Insert current log */
EXEC xp_readerrorlog 0

/* Insert previous log */
--EXEC xp_readerrorlog 1

WHERE SSEL.Text LIKE '%Backup%'

The only issue with this is that you need to find out how many log files you need to insert into the temporary table (if you want to search through all available logs). But you can also find the answer to that question with a query.

Number of configured logs
You can configure the amount of error logs from SSMS. If you right-click on the SQL Server Logs in the object explorer, you can click on Configure:

In the pop-up window, you can configure the amount of archives:

If you script out that change, you’ll end up with this script:

USE master

EXEC xp_instance_regwrite

So you can use the same method (but in reverse of course) to find the amount of configured logs. You just need to use the xp_instance_regread (blogged about this before) stored procedure:

DECLARE @NumErrorLogs int

EXEC master.dbo.xp_instance_regread
	@NumErrorLogs OUTPUT

SELECT @NumErrorLogs AS [NumberOfLogFiles]

There are 2 results possible from this:

– NULL: This means the setting is not configured, and the default (6 archives) is used
– An integer between 6 and 99: This means the amount of logs is configured and not set to default. The number that is returned is the amount of archives

This result can be used to configure the amount of inserts in the script posted above. If a NULL is returned, you know you need to use the default setting of 6. If an integer is returned, you can use that number to determine the amount of archives you’d like to use.

By retrieving the SQL Server error log with a T-SQL query, it’s easy to automate this process if needed. For example, you might want to store the error log data in another place, and not keep it stored in the log files on your instance. You can do that by modifying this script, or write your own solution. By using the xp_instance_regread and xp_readerrorlog or sp_readerrorlog, you can be sure your script doesn’t fail or misses some data.

Where to find backup- and restore history

Interesting questions don’t always need to be hard to answer. Earlier this week I was asked “How can I see what backup was restored on environment X?”. The answer is as interesting as the question…

To see what backups are taken, you can take a look at the history of your maintenance job (depending on whatever type you use in your environment). But there is another way: just ask SQL Server.

First, let’s create a new database named Sandbox (before you run the scripts in this blog post, change or create the directories that are used!):

USE master

	ON PRIMARY (NAME = N'Sandbox', FILENAME = N'C:\Database\Sandbox.mdf', SIZE = 4096KB, FILEGROWTH = 1024KB)
	LOG ON (NAME = N'Sandbox_log', FILENAME = N'C:\Database\Sandbox_log.ldf', SIZE = 1024KB, FILEGROWTH = 1024KB)

Before we start to backup this database, I want to make sure the information I see is only for this backup. In order to do so, I’m going to cycle the SQL Server error log, and clean all my backup history from msdb. You can do that by running these statements:

EXEC sys.sp_cycle_errorlog
EXEC msdb.dbo.sp_delete_backuphistory @oldest_date =  '2016-01-01'

By setting the “@oldest_date” to a date in the future, all backup history will be deleted.

Now that we’ve done that, let’s take a backup of our database:

 TO DISK = N'C:\Database\Backup\Sandbox_2010828_0832.bak' WITH STATS = 10;

The backup will complete in an instant, because there is no data to backup:

The result of the backup can be found in 2 locations. The first one is the SQL Server error log:

EXEC xp_readerrorlog

If you search in the error log you can find the exact time the backup was taken, the amount of pages that were processed, the location of the backup file, etc (image clickable for larger version):

The SQL Server error logs are also available from the SQL Server Management Studio (SSMS). If you connect to your instance with the object explorer, and navigate to Management -> SQL Server Logs, you can find the same information.

The second location you can use to retrieve this information is msdb. The msdb contains the history of your backups:

You can retrieve this information by running a query on the backup-tables:

	CASE B.type
		WHEN 'D' THEN 'Database'
		WHEN 'L' THEN 'Log'
	END AS BackupType
FROM msdb.dbo.backupset B
INNER JOIN msdb.dbo.backupmediafamily M ON M.media_set_id = B.media_set_id

The information for restored backups can also be found in the same 2 locations. Let’s take a backup of the Sandbox database first:

USE master

FROM DISK = N'C:\Database\Backup\Sandbox_2010828_0832.bak' WITH REPLACE, STATS = 10

If we look at the SQL Server error log again we can find the restore time, used backup, etc:

We can also query the msdb again. But this time, we need to use the restore-tables:

FROM msdb.dbo.restorehistory H
INNER JOIN msdb.dbo.restorefile F ON F.restore_history_id = H.restore_history_id

This shows you all recent restores on your instance:

Being able to retrieve the information you need to determine which database was backed-up or restored can help you solve some weird issues. In my case, it explained why a certain record wasn’t restored on environment X: it was inserted after the backup was taken. Issues like that can be quite hard (or even impossible) to find without information about backups and restores.

Time traveling with SQL Server 2016: Temporal tables

Since the release of the first publicly available SQL Server 2016 CTP2, we have a whole list of new and cool features we can start using. One of those features is called temporal tables (or system-versioned tables). So what are temporal tables, and how can you use them?

What is a temporal table?
To understand the functionality of temporal tables, we first need to agree on the definition of a table in SQL Server. If you run a query on a table that contains data, that query can only return the current version of a record. This means you can only see the current “truth”, and there’s no way to travel back in time to see older versions of this record.

Temporal tables give you the opportunity to time travel in SQL Server. For every data change (Update, Delete and Merge) in your table, a historical copy is stored in a history table. The table on which you enabled this data tracking is changed to a “System-Versioned” table.

How does it work?
For every update or delete on a table where system_versioning is enabled, a historical copy of the record is stored:

The original image can be found in this blog post, written by Manoj Pandey (Blog | @manub22).

But how does that work? Let’s start by creating a table we will use to test system-versioned tables:

CREATE TABLE dbo.BankAccount
	(AccountNumber INT CONSTRAINT PK_BankAccount_TransactionID PRIMARY KEY,
	 AccountBalance FLOAT);

In order to make a table a system-versioned table, we need to add 2 datetime2 columns to our newly created table. SQL Server will use these columns to store a from- and to-date that will indicate the valid period of time of the record. By telling SQL Server to use these 2 columns with the keywords “PERIOD FOR SYSTEM_TIME”, SQL Server knows that it needs to use these columns when querying data from the history table:

ALTER TABLE dbo.BankAccount

In the above script, the “HIDDEN” keyword is used. This can only be used from CTP 2.1 and higher. What HIDDEN does, is hide the columns in your original table to make sure application- and database code isn’t going to fail because you added these 2 new columns. You can still query the columns, but if you run a “SELECT *” query these columns won’t be returned.

When that’s done we can enable system-versioning on the table:

ALTER TABLE dbo.BankAccount

Data changes & history
Now we can take a look at how data changes are captured, and history is recorded. Before you execute any of the statements below, enable the execution plan in your SSMS. If we insert a new record, no history is created:

INSERT INTO dbo.BankAccount
	(AccountNumber, AccountBalance)
	(2147483647, 10000)

We just see the expected Clustered Index Insert, nothing special:

Now, let’s update the bank account balance (the WAITFOR is added just to make sure we have a clear difference in change time for the next few steps):

UPDATE dbo.BankAccount
SET AccountBalance = AccountBalance - 2500

WAITFOR DELAY '00:01:00'

UPDATE dbo.BankAccount
SET AccountBalance = AccountBalance + 1000

If you look at your execution plan, you’ll see an extra DML command in the execution plan:

Besides the expected Clustered Index Update, there is an extra Clustered Index Insert. This is the insert statement that is generated by SQL Server, to insert a record in the history table. There’s no user interaction needed to store historical data. SQL Server will take care of that process for you.

Time travelling
Now that we store historical data, it’s time to start the actual time travelling. To paint a complete picture, this is a timeline of the data inserts and updates:

Now that we have a clear timeline, let’s start our time travel. Besides creating and maintaining the history table, you also get to use a bit of new syntax in your query. For system-versioned tables, you can use the “FOR SYSTEM_TIME” clause in your query. In order to show you the timeline in SQL Server, let’s use this new syntax to query our history per minute:

SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF '2015-07-09 09:42:00' --No data available
SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF '2015-07-09 09:43:00' --First insert
SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF '2015-07-09 09:44:00' --First update
SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF '2015-07-09 09:45:00' --Second update
SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF '2015-07-09 09:46:00' --No other changes

Remember, this is based on my data. So if you want to query your timeline, change the datetime string based on your data!

So basically, you can start querying your data by using a specific point in time:

SET @PointInHistory = '2015-07-09 09:45:00'

SELECT * FROM dbo.BankAccount FOR SYSTEM_TIME AS OF @PointInHistory

Besides the actual point in time, you can use 3 other ways to query your data:

SELECT AccountNumber, AccountBalance, ValidFrom, ValidTo
FROM dbo.BankAccount
FOR SYSTEM_TIME FROM '2015-07-09 09:43:00' TO '2015-07-09 09:45:00'

SELECT AccountNumber, AccountBalance, ValidFrom, ValidTo
FROM dbo.BankAccount
FOR SYSTEM_TIME BETWEEN '2015-07-09 09:43:00' AND '2015-07-09 09:45:00'

SELECT AccountNumber, AccountBalance, ValidFrom, ValidTo
FROM dbo.BankAccount
FOR SYSTEM_TIME CONTAINED IN ('2015-07-09 09:43:00', '2015-07-09 09:45:00')

Table changes
In practice you’ll not only see data changes happening on tables, but you’ll also have schema changes on your table once in a while. But how does a temporal table cope with schema changes? Let’s try to add a column to our table:

ALTER TABLE dbo.BankAccount ADD NewColumn VARCHAR(10)

This results in an error:

In order to add a column, we need to disable the system-versioning first, before we add the new column:

ALTER TABLE dbo.BankAccount ADD NewColumn VARCHAR(10)

Now let’s enable system-versioning again:

ALTER TABLE dbo.BankAccount

This results in another error:

So we first need to add the same column to the history table, before enabling system-versioning again:

ALTER TABLE dbo.MyBankAccountHistory
ADD NewColumn VARCHAR(10)

ALTER TABLE dbo.BankAccount

Disabling system-versioning means that for a short period of time, you could loose historical data. A possible way to get around this, is by setting your database in single user mode before you disable it. This prevents other user settings from changing data in that table. If you run this in your production environment, this might not be your best option though! By disabling system-versioning, historical data will not be removed.

Indexing historical data
Performance isn’t only important when you query your “normal” tables, but also if you want to query your newly created temporal tables. If you want to write a business process that uses your historical data, you need to make sure this doesn’t slow down your application. But how do you achieve that?

Just like on a normal table, you can create indexes no the history table. This table is basically a normal table, so there’s no reason why you can’t create indexes on it:

ON dbo.MyBankAccountHistory (AccountNumber, AccountBalance)

Historical data cleanup
It’s really great to store historical versions of your records in a separate table, but this also provides you with new challenges. You need more storage for your data, and you might want to remove data after a certain period of time. At this point in time, system-versioned tables don’t have a cleanup process. This might change in the future, but right now you need to cleanup these tables yourself. So if you want (or need) to cleanup old data, you need to write a small process yourself. This can be as easy as writing a query to run through all system-versioned tabled and delete old data, and schedule that in a SQL Server Agent job. But again, this is something to keep in mind when you start working with temporal tables.

I had a really good conversation about this with Borko Novakovic (@borko_novakovic), who is the Program Manager for this part of SQL Server. The code below is an example of such a process, that Borko provided:

    /*Drop versioning to allow deletion of historical data*/
       ALTER TABLE dbo.BankAccount
       /*DELETE MyBankAccountHistory table*/
       FROM dbo.MyBankAccountHistory
       WHERE ValidTo < '2015.01.01'
       /*Re-establish versioning*/
       ALTER TABLE dbo.BankAccount
                  (HISTORY_TABLE  = dbo.MyBankAccountHistory,
                   DATA_CONSISTENCY_CHECK = OFF));

Deleting data without disabling system-versioning doesn’t work, because you’ll get an error like this:

I do want to thank Borko again for his time. He helped me out a lot, and I’m thankful he took the time to do that. Thanks Borko!!!

Remove system-versioning
Removing system-versioned tables from your systems takes one extra step, because just executing a “DROP TABLE” statement doesn’t work. SQL Server will prevent you from dropping temporal tables, without you first disabling it. Let’s start by disabling system-versioning:

This disables the system-versioning on your original table, and transforms the history table to a normal table. If you want to get rid of the tables, you can just drop them both like you would normally do:

DROP TABLE dbo.BankAccount
DROP TABLE dbo.MyBankAccountHistory

Besides this blog, there are 2 fantastic posts you need to read about this topic, written by Itzik Ben-Gan (Blog | @ItzikBenGan), that you can read at SQLMag.com (Part 1, Part 2).

One thing you definitely MUST SEE is a video with Borko and Scott Klein (Blog | @SQLScott) on Channel 9. In this video, Borko explains the idea behind temporal tables, and shares some insights on this new feature.

Other information on temporal tables can be found on MSDN:

Temporal Tables
SQL Server 2016 (MSDN root)

Source control: Using Visual Studio Online in SSMS

As database professionals, we’re all aware of the importance of backups. We make sure the backup process runs as expected, and (hopefully) we restore a backup every now and then to test if the restore process works. But what about source control for your scripts? Most of us don’t use source control in our daily job. But source control is a much a part of the backup process as the actual database backups.

So looking for an easy way out, I focused on Visual Studio Online (VS Online). This is a free online source control system, that you can use once you created an account. You can login on the website with your Microsoft Live account, enter some information (like a username, etc), and you’re ready to go!

But how do you go from writing a query in SQL Server Management Studio (SSMS), to checking in your .sql code files into VS Online? Let’s take a look.

Installing TFS tools
In order to use VS Online, you need to install some extra things on your machine. The first thing you need to download is the “Team Explorer for Microsoft Visual Studio 2013”. This installs the team explorer, that you need to get things from and check thing into VS online. This install requires a reboot, so please take that into account!

The second and last installation is the “Microsoft Visual Studio Team Foundation Server 2013 MSSCCI Provider”. This installs the provider, that takes care of the communication with VS Online.

For this installation, it’s important that you download the right version. If you’re running the 32-bit version of SSMS, you need to download the 32-bit version of the provider. If you install the 64-bit version, it won’t work (and trust me on this, I’ve made that mistake before!).

VS Online
At this point, I’m assuming you have an account for VS Online. If you log in to the website, and you go to your account url ([AccountName].visualstudio.com), you see the “Create your first team project” page. On this page, you need to create a project before you can check in any files:

Configure source control in SSMS
The next step is to configure SSMS, to use the source control provider you just installed. After the installation, you get a new menu in SSMS for source control:

In this menu, click on “Open from Source Control”. In the window that opens, click “Servers…”:

Click “Add…” to add a TFS server or URL:

Now you need to enter your account URL in the textbox:

Once you’ve done that a login screen pops up. Log in with your Microsoft Live account (the same you used to create your VS Online profile), and you’re authenticated:

At this point, your source control is added, and you’ll see it in the overview screen:

If you close the windows that are shown, you end up in your source control project overview:

Creating a SQL Server Scripts project/solution
Now that we’ve installed the source control providers, it’s time to create our SQL Scripts project. In SSMS, click on File => New => Project (or Ctrl+Shift+N), and choose “SQL Server Scripts”:

Now that you have a project, it might be handy to show what’s in the solution (if you don’t have that on your screen already). You can open the “Solution Explorer” by clicking on View => Solution Explorer (or Ctrl+Alt+L).

Checking in your solution
If you want to check-in your solution or project, you can right-click it in the Solution Explorer, and choose “Add Solution to Source Control”:

Another way of doing it, is by clicking on File => Source Control => Add Solution to Source Control. In the next window you are asked for the source control server (click okay if your VS Online URL is selected), and you see the project explorer:

Click on “MySQLProject”, and click OK. Now the check-in screen pops up, where you can “tag” your check-in:

Congratulation! You just checked in your first code in VS Online!

Pitfalls and difficulties
In all honesty, there are some downsides in using this, but those are mainly issues in SSMS. For example, you can’t create folders in your SQL Script project. You have to work with the default “Queries” folder you get by default. So there’s no chance you can add a “Finance”, “Maintenance” or “DBA” folder to your project for example.

A work-around for this is creating multiple projects in the solution. So you’ll end up with 3 projects in 1 solution. The nice thing is that you can check-in or check-out 1 single project in your solution, or you can just work on the entire solution at once. This gives you the possibility to work on the solution on your own, or with a group of people.

Another pitfall is the process. If you’re not used to working with queries in source control, it’s easy to forget to check-in your changes at the end of the day. But that’s just something you need to get used to, and need to deal with in your own way. If you’d like to check-in multiple times a day, go ahead and do that. Do you want to check-in your changes at the end of the day only, it’s fine as well.

Using source control is great, especially when you have a lot of scripts you need to maintain. I’ve tried to create zip-archives, version numbering my .sql files, different directories, etc. But there’s always a moment that you forget to save your script in an archive, or your computer crashes, and all your work is gone. Using source control prevents these issues (if you use it as intended of course!). VS Online is a really good source control platform (it’s basically TFS online), and it’s free for use. One of the mayor advantages is that VS Online / TFS is fully compatible with SSMS and Visual Studio (which you both use as database professional).

Are you still skeptical? Maybe you should just try it out for a week, to see if it helps you in your daily job. And you know what, maybe you’ll like it. And even better: maybe it’ll someday save your life!

On which port is SQL Server running?

SQL Server Error Log

xp_readerrorlog 0, 1, N'Server is listening on'

The result:

SQL Server Configuration Manager

You can also query the system DMV’s:

FROM sys.dm_exec_connections
WHERE local_tcp_port IS NOT NULL

SELECT local_tcp_port
FROM   sys.dm_exec_connections
WHERE  session_id = @@SPID

Command prompt

“netstat -ano | findstr [PID]”

The result:

        @RegKeyName VARCHAR(8000);

EXEC xp_regread
  @rootkey = 'HKEY_LOCAL_MACHINE',
  @key = @RegKeyName,
  @value_name = 'TcpPort',
  @value = @TCPPort OUTPUT;

Event Viewer


Is it ok to be taking metoprolol and mononessa at the same time?

METROPROLO&MONONESSA. It is ok to take as long as your doctor or cardiologist is aware of it and you are being monitored and is aware of your combined use.It can increase the serum level a little bit by slowing the first pass metabolism of Metroprolol and other beta blockers.It may need dose adjustment. But i will suggest you inform your doctor who prescribed the drug metroprolol.


The Affect of Selection in Digital Sound Art

1A variety of digital technologies exist that can transform sound waves into binary code. Any sound can be ‘sampled’ in this way: drum beats, field recordings, even full-length instrumental or vocal tracks. Copies are then ready to be duplicated, cut, spliced, pitch-shifted, equalised, turned backwards, and/or passed through a myriad of different effects processors before being sequenced and layered into a final stereo mix.

2Once obtained, this mix is itself eminently copy-able without loss in audio quality. Moreover, affordable digital reproduction media abound, such as CD-R and MP3. These methods of distribution are as attractive to the home audio producer as they are to the file-sharing enthusiast. On one end of the spectrum one finds an increasing number of people downloading music and creating personalised mix CDs or play-lists, irrespective of the constraints of copyright. On the other end, one observes established pop musicians such as Björk writing original choral arrangements, only to sing them into hard-disk recording systems for further remixing prior to commercial release (as on her last album Medúlla (2004)). Somewhere in the middle lies DJ Danger Mouse’s The Grey Album, an unsanctioned (and widely downloaded) remix of the Beatles White Album (1968) mashed together with a-cappella raps from Jay Z’s The Black Album (2003).

3It is as though audio in the digital age has become an ‘utterly malleable material’ (Miller 20), leaving only the sound selector and manipulator as the all-powerful source of creativity, musical genius and/or surreptitious intent (in the case of file sharing). To date, the majority of academic work on digital sound practices relies upon this suggestion that technology has provided us with almost total control over recorded sound (see Miller, Mudede, Pinch and Bijsterveld, Lysoff, Cascone, Théberge, and Jones).

4While digital methods for copying audio have been available for over 20 years, the technology involved has far from exhausted its potential to open up new forms of musical composition, cultural expression, and social exchange. In fact, the practices it has enabled are so multifaceted that the only limit in this case appears to be our imagination. This begs the question as to what limits this imagination, if anything?

5Do the sounds we mix not require us to attend to their particular shape before we bring them into alignment with other sources? What techniques must we employ to make room for their specific timbres? What constraints lead to the adoption of these techniques over others? How and why are these sounds chosen in the first place? What are the resources or archives used?

6It is through such questioning that the ‘give and take’ involved in digital audio production becomes apparent. The technologies used are not simply vehicles of personal expression – they represent an entire family of practices employed in a wide variety of unique scenarios. No two situations will be exactly alike, but there will be resemblances. These similarities and/or differences are as much the result of constraints prescribed by one’s sonic starting-points as they are the product of individual style. Such a symmetrical perspective is paramount in the development of analyses that accurately reflect the heterogeneity of digital audio practices. In the rest of this article I attend to the production methods of three Montreal-based sound artists. Quotations from personal interview are contrasted against the dominant academic view of digital audio as a vehicle of subjective control over sound.

7Jennifer Morris (a.k.a. ‘[sic]’) works mostly with field recordings as well as sounds that she collects by close-micing instruments such as the guitar or Japanese koto. When asked about how her sonic source material affects her work, Morris’ answer reflects a deep respect for the individual character of the sounds she chooses to copy and remix. At the same time she is not afraid to alter samples in search of new experimental mixing trajectories. In her words:

My work is composed from the source material. It is a very organic process that begins with the selection of a few sounds and usually ends with using none of them and sounding completely different from what I set out to do. That’s what I love most, the sources control the work completely. (Jennifer Morris, a.k.a. ‘[sic]’, interviewed by Owen Chapman, 14 Dec. 2004)

9Morris habitually begins her compositional process with short copied extracts of longer recordings that she then transforms with the aid of various pieces of sound software and hardware to the point where they are barely recognisable. She will often discard her initial sound-selections as new timbres emerge through her experimentation. Nevertheless, Morris claims that this process is guided by her sources completely. In so doing she ascribes the latter a type of agency. There is a reverence in her words even in the way she describes the task of collecting sounds for future use. Instead of copying audio that fits the musical plans she has in her head, Morris waits for sounds to set off her ‘sampling radar’, so to speak. As she says about the sounds she gathers in her field recording practice, ‘Even if they [cannot be] heard by the human ear, I try to figure out ways to capture them, with a pickup and such.’ The sounds she selects are then allowed to guide her treatment of them based upon their own characteristics, such as sonic density, frequency range and mood.

10For Anna Friz (a.k.a. ‘A2Z’), however, the material she copies when starting a new sound piece sometimes ends up driving the work, and at other times not. When asked about the impact of her initial sound selections upon her final product, she answered:

I think that sometimes it is the indexical relationship that the samples have with where they came from that … you’re drawn to and that you want to use, you want to emphasize. And in that case I think that some of the samples do make some decisions for you. Because there are certain things that, you know, depending on who you are you would choose not to do to a sample. (Anna Friz, a.k.a. ‘A2Z’, interviewed by Owen Chapman, 9 Sept. 2004)

12In elaborating this point Friz mentions a particular recording she possesses from the clash between police and protesters at the APEC summit in Vancouver in 1999. A member of staff from the radio station where she was working (CiTR) had called from the protest on a cell phone and was describing the pandemonium that was breaking out all around him. The urgency and panic in his voice as he shouts ‘They’re advancing right now…and we’re running, we’re running,’ became an artifact that she felt she must protect. As she said: ‘I didn’t want to just plant it into something without a context, because, for me it came from such a political place that I really wanted to preserve that.’ This sound fragment became the foundation for a political piece that she later composed in response to the treatment of protestors at the Quebec FTAA summit (co-produced with Richard Williams and entitled: ‘there’s a risk of arrest if you turn right’ (2001)).

13As with Morris, Friz often ends up radically transforming the sources she starts with. Interestingly, while Morris describes this as following intuitions laid out for her in the original material, Friz tends towards a more subjective interpretation of the mixing process:

[I’ll take] something out of context because it just has a certain feel to it, a texture, or a pitch or tone or something and I just think: ‘That’s a good sound. I could turn that into something else – it will go with these other things.

15While this answer might resemble accounts focused on subjective expression through digital audio, it is important to recognize that Friz has no problem admitting to being impacted by a specific sound before deciding to displace it from its original context.

16In my own case I feel as though my usual source material (musical fragments copied from vinyl records) dramatically affects the work I end up producing. I’ll start by sampling a short passage uncovered on one record and, through working with it, be led towards other parts of my collection in search of other sounds. The process of composition unfolds organically as each additional element reveals hidden aspects of the samples already selected. This is where the affect of selection in digital sound art becomes most apparent, in the unique nature of every juxtaposition or mix. Contrary to predominant opinion, this mixing is as much guided by spontaneous inspiration based on aural-reaction as it is the subjective will of the mixer. This holds for all cases of digital audio manipulation – whether we are discussing studio production, sampling, file sharing or mash-ups, the common feature in all of these practices is the (re)collecting of recorded sounds in a way which, to quote Walter Benjamin, ‘studies and loves them as the scene, the stage, of their fate.’ (Benjamin 60).

17Asserting increased aural control as paramount to users of digital audio technology reduces a complex interactive relationship to one of master and slave. Although extraordinary editing precision is now possible when compared to the days of magnetic tape, copying, splicing and transforming audio with digital equipment is still very much about responsive listening. Those looking to write about the relationship between digital audio technology and contemporary practices of music making must never overlook the affective dimension of sound – its fleeting presence as vibrations of the ear drum. For recorded audio, no matter how many times it has been copied, it must always pass through a singular moment of conversion into moving air molecules in order to be experienced – at which point there is nothing left for us to do but listen and learn.


News Home

Why the coal industry is right to fear divestment

Opinion The Drum
The divestment campaign is currently targeting universities with large endowments.

The divestment campaign is currently targeting universities with large endowments.

Divestment by Australia's Group of Eight universities would have little direct impact on the fossil fuel industry, but the message it would send would be powerful, writes Ben Caldecott.

In July 2012, Rolling Stone magazine published an article by Bill McKibben titled Global Warming's Terrifying New Math. In order for the world to have an 80 per cent chance of keeping global warming below 2 degrees Celsius, he called for divestment from fossil fuel companies to "spark a transformative challenge to fossil fuel ... [by] moral outrage".

Inspiration for the fossil fuel divestment idea leans heavily on the perceived success of the South Africa divestment campaign in the 1980s in putting pressure on the South African government to end apartheid.

Divestment campaigns typically evolve over three waves. The first wave begins with a core group of investors divesting from the target industry. Previous divestment campaigns have tended to originate in the United States and in the first phase focus on US-based investors and international multilateral institutions. The amounts divested in the first phase tend to be very small but create wide public awareness about the issues.

Both in the case of tobacco and South Africa, the campaigns took some years to gather pace during the first wave until universities such as Harvard, Johns Hopkins and Columbia announced divestment in the second phase. Previous research typically credits divestment by these prominent American universities as heralding a tipping point that paved the way for other universities, in the US and abroad, and select public institutions such as cities to divest as well. In the third wave, the divestment campaign can go global and begins to target very large pension funds and market norms.

Like all previous divestment campaigns, the fossil fuel divestment campaign has started in the US and in the short term focused on US-based investors. Over the last year, the campaign has attempted to build global momentum by targeting other universities with large endowments such as the universities of Oxford and Cambridge in the UK and the Group of Eight universities in Australia.

From the perspective of the three waves of divestment, the fossil fuel campaign has achieved a lot in the relatively short time since its inception: 10 colleges and universities have committed to divest, along with 22 cities, two counties, 23 religious institutions, 19 foundations and six other institutions. The campaign can be said to be entering the second wave of divestment.

Of the $12 trillion assets under management among university endowments and public pension funds - the likely universe of divestment candidates - the plausible upper limit of possible equity divestment for oil & gas companies is in the range of $240-$600 billion and about another half that for debt.

University endowments in the US have 2-3 per cent of their assets committed to fossil fuel stocks, while the proportion in the UK is higher with an average of 5 per cent largely because the FTSE has a greater proportion of fossil fuel companies. Public pension funds, likewise, have 2-5 per cent of their assets invested in fossil fuel related public equities.

In recent research we have completed at the University of Oxford's Smith School of Enterprise and the Environment, we find that the direct impacts of fossil fuel divestment on equity or debt are likely to be limited. The maximum possible capital that might be divested by university endowments and public pension funds from the fossil fuel companies represents a relatively small pool of funds. Even if the maximum possible capital was divested from fossil fuel companies, their shares prices are unlikely to suffer precipitous declines.

But even if the direct impacts of divestment outflows are meagre in the short term, our research shows that a campaign can create long-term impact on the value of target firms through a process of stigmatisation. The outcome of this stigmatisation process, which the fossil fuel divestment campaign has triggered, poses a far-reaching threat to fossil fuel companies and the vast energy value chain. Any direct impacts of divestment pale in comparison.

As with individuals, a stigma can produce negative consequences for an organisation. For example, firms heavily criticised in the media suffer from a bad image that scares away suppliers, subcontractors, potential employees, and customers. Governments and politicians prefer to engage with 'clean' firms to prevent adverse spill-overs that could taint their reputation or jeopardise their re-election. Shareholders can demand changes in management or the composition of the board of directors of stigmatised companies. Stigmatised firms may be barred from competing for public tenders, acquiring licences or property rights for business expansion, or be weakened in negotiations with suppliers. Negative consequences of stigma also include cancellation of multibillion-dollar contracts or mergers/acquisitions. Stigma attached to merely one small area of a large company may threaten sales across the board.

One of the most important ways in which stigmatisation could impact fossil fuel companies is through new legislation. In almost every divestment campaign we reviewed from adult services to Darfur, from tobacco to South Africa, divestment campaigns were successful in lobbying for restrictive legislation affecting stigmatised firms.

As fiduciaries - managing long-term savings on behalf of their beneficiaries - endowments, pension funds and similar institutional investors have a duty to understand and respond to challenges posed by the fossil fuel divestment campaign, whether considering fossil fuel divestment or not. The divestment campaign could pose considerable reputational risk to fossil fuel companies even if its immediate direct effects are likely to be limited.

Ben Caldecott is the director of the Stranded Assets Programme at Oxford University's Smith School of Enterprise and the Environment. View his full profile here.

More Stories


digestiveAs everyone is aware, our digestive systems are filled with bacteria, and this is a good thing. Despite what all the soaps and solvents would have you think, bacteria are very necessary for optimal health. As a matter of fact, without this ecosystem of bacteria, we would surely die. But a recent study published in the journal Nature indicates that many of us don’t only not have the proper balance of “good” bacteria in our digestive systems, we have less than what is needed to “maintain good health.”

The study from officials with the Faculty of Health and Medical Sciences at the University of Copenhagen estimates about one-quarter of the population has 40% less intestinal bacteria than what is considered healthy. This is extremely important, as gut flora is said to account for nearly 80% of our immunity, protecting us from brain decline, the seasonal flu, and even protecting our bodies against cancer.

“Not only has this quarter fewer intestinal bacteria,” said lead researcher Oluf Pedersen, “but they also have reduced bacterial diversity and they harbor more bacteria causing a low-grade inflammation of the body.”

The study was conducted on 292 people from Denmark, 169 of which were obese and 123 of normal weight. The researchers indicated the overweight and obese were more likely to have this detrimental low gut bacteria.

“Our study shows that people having few and less diverse intestinal bacteria are more obese than the rest. They have a preponderance of bacteria which exhibit the potential to cause mild inflammation in the digestive tract and in the entire body, which is reflected in blood samples that reveal a state of chronic inflammation, which we know from other studies to affect metabolism and increase the risk of type 2 diabetes and cardiovascular diseases.”

Interestingly, another study from a French research team found that those overweight patients with low gut bacteria were able to increase their intestinal bacteria after following a 6-week low-fat diet. As low-fat diets are usually rich in plant-sources, the unprocessed and natural food sources no doubt helped to change the bacterial balance.

“This indicates that you can repair some of the damage to your gut bacteria simply by changing your dietary habits,” said Prof. Pederson. “Our intestinal bacteria are actually to be considered an organ just like our heart and brain, and the presence of health-promoting bacteria must therefore be cared for in the best way possible.”

This means you can make significant and nearly-immediate changes to ensure your intestinal bacteria are balanced and therefore health-promoting. This is done by eating natural, organic produce and limiting processed foods. Adding fermented foods like kimchi, miso, and kefir are also great sources of probiotics, or good bacteria.

Additional Sources:

Storable Food


To Top

The British Museum VS Prince William over ivory trade

The British Museum is arguing that the Duke of Cambridge’s support of a total ban on ivory trade would harm its collection of “indispensable” treasures.

The UK government is currently considering a UK-wide ban on the sale of all carved ivory objects. Currently, only modern objects made after 1947 are illegal. The new legislation would mean that all ivory made objects would be illegal.

Experts, however, have opposed this legislation, with the British Museum claiming that there was no “public benefit” in restricting the display and transport of ivory art made before 1947. Some, such as historian David Starkey, claim that a total ban would essentially be “cultural vandalism.”

The ivory collection in the British Museum includes the 3,000-year-old Nimrud Ivories which were excavated in the mid 20th century, as well as the Benin Ivories which date from the 15th and 16th century. Ivory pieces “are integral pieces of the collection and play an indispensable part in the museum’s presentation of the history of human cultural achievement,” a spokesman for the British Museum said.

The British Museum was keen to emphasise that they “support any efforts to protect elephants in Africa and Asia and kerb the illegal trade and export of ivory, but we are also clear that this should not include antique ivory works of art. There is no public benefit in restricting the display or movement of ivory works of art made before 1947, and legislation should not extend to cover actions carried out before this date.”

Prince William is a strong conservationist and has supported similar bans in the past. Last November, William spoke at an international conference on the illegal wildlife trade that “ivory treated as a commodity is the fuel of extinction.”

“So the question is: why are we still trading it? We need governments to send a clear signal that trading in ivory is abhorrent.”

The ivory trade, while still massively common, has come under attack from countries where the trade plays a large part. Most recently, China said that they would ban all ivory trade by 2017; it is estimated that 70% of the world’s trade ends up in China.

It is clear that the view of Prince William is at odds with those expressed by the British Museum.

  • Kathleen Ames

    I think it is futile to destroy ancient treasures whether ivory or other material.

  • robert

    African elephants are living treasures. At the rate humans are killing them, they will be extinct in about twenty years. Elephants are infinitely more precious than the trinkets made from their tusks. God bless the Duke of Cambridge.

More in The Cambridges


Oversized roll-neck sweater

Demna Gvasalia takes an avant-garde interpretation of femininity for Balenciaga's AW17 collection and this charcoal-grey wool-blend sweater offers a particularly cool interpretation. The exaggerated silhouette is knitted in Italy with a chunky V-neckline cut-out detail across the side hems then softly gathered from the back yoke which is detailed with a white embroidered logo. Try it tucked into one of the house's vibrant skirts.


American Nuclear Society

Home / Publications / Journals / Nuclear Science and Engineering / Volume 172 / Number 3

Statistical Theory of Fission Chains and Generalized Poisson Neutron Counting Distributions

Manoj K. Prasad, Neal J. Snyderman

Nuclear Science and Engineering / Volume 172 / Number 3 / November 2012 / Pages 300-326

Technical Paper /

The neutron counting probability distribution for a multiplying medium was shown by Hage and Cifarelli to be a generalized Poisson distribution that depends on the fission chain number distribution. An analytic formula is obtained for this number distribution, the probability to produce a number of neutrons in a fission chain. The formula depends on the probability that a fission spectrum neutron induces a subsequent fission and depends on the probability distribution for a specific number of neutrons to be produced in an individual induced fission. The formula is an exact solution to a functional equation due to Böhnel for the probability generating function. The Böhnel equation is derived as the t [right arrow] limit of a rate equation for a neutron population generating function, related to a rate equation studied by Feynman. The Böhnel equation is also shown to be a fixed point of an iteration problem, related to one studied by Hawkins and Ulam, where the iteration generates the chain a generation at a time. The discrete iteration problem is shown to be connected to the continuous time evolution of the chain. An explicit solution for the time evolution of the chain is given in the simplified approximation where at most two neutrons are created by an induced fission. The t [right arrow] limit of this equation gives a simple analytic expression for the solution to the Böhnel equation in this approximation. A generalized Poisson counting distribution constructed from the theoretical fission chain probability number distribution is compared to experimental data for a multiplying Pu sample.

Questions or comments about the site? Contact the ANS Webmaster.


Who Are Direct Care Professionals (DCPs)?

Direct Care Professional (DCP) is the state of Iowa’s umbrella term for individuals who provide “services, care, supervision, and emotional support to Iowans.”  The state’s definition does not include licensed professionals such as registered nurses or licensed social workers.

DCPs provide hands-on personal care and support for children and adults of all ages in a variety of settings.  These settings include homes, nursing homes, hospitals, assisted living communities, hospices, adult day centers, residential facilities, pre-vocational services, supportive employment, and other environments.  DCPs go by many titles, for example, Direct Support Professional; Certified Nursing Assistant (CNA); Medication, Rehabilitation, Restorative, and Hospice Aide; Patient Care Technician; Personal Care Assistant; and some 30 other titles.

In this toolkit we will use Direct Care Professional (DCP) to refer to all those mentioned above.

Mentor Defined

The term “mentor” can mean different things to different people.  Many find it difficult to distinguish among on-the-job training, a preceptor program, employee orientation and the role of a supervisor.  For clarification we offer these definitions:

1. DCP Mentor:  A trusted counselor, guide, coach, or tutor in a long-term, on-going relationship with another DCP.
2. DCP Preceptor:  Also a trusted counselor, guide, coach, or tutor, but for a short-term or one-time relationship with another DCP.
3. Employee orientation:  A new employee orientation is typically a list of policies and procedures that new employees need to be aware of in order to perform well in their jobs, comply with company/organization policy, and ensure their safety and well-being, e.g., fire safety, payroll schedules, tours of various departments, amenities, and parking.
4. On-the-job training:  Training given to employees as they perform everyday work activities.  Training typically consists of observation and participation in job skills demonstrated by experienced workers until the employee demonstrates competency and can work independently.
5. Supervisor:  An individual who reviews job performance, may provide coaching and guidance, but is not considered a peer and is often not a DCP.

Introduction: Page 2


Distillation of Deepnets

Training modern deepnets can take an inordinate amount of time even with the best GPU hardware available. Inception-3 on ImageNet 1000 using 8 NVIDIA Tesla K40s takes about 2 weeks (Google Research Blog).

Even when a large network is trained successfully, the memory footprint and the prediction latency (due to the number of its parameters) can make it challenging to put it into production.

One way to keep the predictive accuracy of a large network but reduce the number of its parameters, is a training paradigm called "distillation". The concept of distillation, introduced by Hinton, Vinyals, Dean, tries to transfer knowledge from a larger network to a much smaller network for deployment.

So what exactly is distillation? In supervised learning techniques, you train the input data on hard targets. If you’re training a network to recognize images of hand written digits, the hard targets will look like the following:

0 1 2 3 4 5 6 7 8 9
0 1 0 0 0 0 0 0 0 0

When the cumbersome model is a large ensemble of simpler models, we can use an arithmetic or geometric mean of their individual predictive distributions as the soft targets.

Hinton, et al. suggest that the class probabilities of of the incorrect categories are still valuable information to assist smaller networks.

A scatter of 100 forward passes of the softmax input and output MNIST. On the XX axis is a rotated image of the digit 1 (Gal)

Training a much smaller network with less data on these soft targets shows promise. This means that once we have our large model trained for months, we can distill the knowledge using the soft targets into a smaller model for deployment in a reasonable time and obtain respectable results.

Just as a review, softmax, is an activation function you can apply to the final layer of your network to turn scores into categorical probabilities. More precisely, given the logits, zi, the inputs to the softmax function, the softmax is defined as follows:

It’s easiest to think of the softmax as the normalized exponential function; the outputs, qi add up to 1, hence turning scores, zi into categorical probabilities.

For distillation, we think of the generalized softmax function which has a temperature parameter T:

When T = 1, this is just the regular softmax function described above. By raising the temperature one can dampen the probability distribution of the classes, in essence “softening” the target categories. You can see this in the example of classifying animal images where the soft targets have a less extreme probability distribution than the ensemble targets or the hard targets.

When transferring the knowledge via distillation from a larger network to a smaller network, you do so by creating soft targets from the larger network with some high temperature, and use the same temperature to train the smaller network. So let vi be the logits to the softmax for the larger network, with the softmax function p. The learning happens by minimizing the cross entropy loss between p and q:

In other words, to minimize the gradient:

If the temperature is high in comparison to the logits, then using a property of exponential functions that e^\epsilon ~ 1 + \epsilon for small enough \epsilon, the above expression can be approximated as

where N is the number of categories. Let’s assume without loss of generality that the logits for the larger network and the smaller network both have mean 0. Then the above is equivalent to

Note that in training, you scale the gradient of the cross entropy by the square inverse of the temperature.

How does using soft targets do in practice? Let’s start with the first problem of distilling a large cumbersome model’s knowledge into a smaller one for easier deployment. Of the examples discussed in the distillation paper, we focus on MNIST. Turns out, using soft targets is actually so good that the smaller model can be trained and generalized to predict classes that it hadn’t even seen before.

As a baseline, using all 60,000 training cases on a large model with two hidden 1600 rectified linear hidden units (ReLU) with dropout, jittering inputs and weight constraints gives 67 test errors. Now using a smaller network with no regularization, specifically just using vanilla backprop in a 784->800->800->10 network with ReLU as activation, gives 146 test errors. Turns out, adding simply the soft targets with a temperature of 20, but with no jittering of inputs or dropouts, results in minimizing the test errors to 74.

Distillation is a technique that tries to simulate the output from a large cumbersome model via a simpler model. Traditionally, the large cumbersome model is a several layered deep net with thousands of units and the simpler model contains, an order of magnitude smaller, number of layers and neurons. While this technique allows us to deploy simpler models in production systems, it usually has a higher error than the cumbersome model. Focusing on the classification case distillation is achieved by training the simpler model on the class probabilities outputted from the cumbersome model.

Learning More

To address some of the issues unique to putting deep learning models into production, Startup.ML is organizing a one-day special conference on Jan 21st in San Francisco. Leading practitioners from Facebook, Yelp, Intel, etc. will cover their approaches to solving these problems in hands-on workshops and talks.

The conference will also include a hands-on tutorial on TensorFlow by Illia Polosukhin one of the most active contributors to Google's popular deep learning library.

195.00 295.00

Gal, Y. "What My Deep Model Doesn't Know..."

Google Research Blog. "Train your own image classifier with Inception in TensorFlow

Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).

Hinton, Vinyals, Dean, "Dark knowledge"

Juan Carlos Asensio

Gurkaran S. Buxi

Arshak Navruzyan


Google+ Followers

måndag 5 december 2016

Unidentified flying object

A floating saucerful of secrets

The vast, last shade of the lamp,
its metal head,
thin white legs.
Stardust from above,
from underneath.

This lonely cloud floating
in a rocky universe,
this flying saucerful of secrets.
Meeting on a timetable
between a polarized planet
and comets and asteroids
with swimmer's tail.

An ark of Noahs and nobodies,
pairing organic and inorganic,
reasons and rubbish,
means of love
and martial arts.

A rescue boat crowded
with all that humanity got left,

boundless opportunities
for every individual
answer an question.

We floating saucerful of secrets,
standing on the top of toes,

with noses barely on surfaces.
If we at least could be
an unidentified flying object
for someone to love or fear.

(sound track: Pink Floyd | A Saucerful Of Secrets)

Inga kommentarer:

Skicka en kommentar