Azure Synapse Analytics has introduced new features for its workspace deployment extension, enabling automated publishing of artifacts from any user branch directly to target environments, bypassing the manual "publish" button in Synapse Studio.
Mind Map
Click to expand
Click to explore the full interactive mind map • Zoom, pan, and navigate
[Music] foreign
and I'm a supporters Coalition engineer
on the Azure synapse analytics team
welcome to this video series on synapse
CI CD
in today's video we will be covering
some new and exciting features recently
delivered by synapsia City team to do
synapse workspace deployment extension
this new features will allow you to
automate the publishing of your
artifacts from any user branch
as you know until recently the arm
template generation could only be
triggered from synapse Studio when using
the publish button
now with these new features you can pick
any user Branch to generate These Arm
templates and how to make this
publishing operation from either your
Azure devops automated workflow or GitHub
GitHub
let's watch the demo and see how you can
achieve this
consider this typical CI CD workflow
where you have multiple developers
working in feature branches and when
they need to merge their changes to the
master Branch they create a pull request
all changes are now part of the
collaboration branch and if they want to
generate the arm templates and publish
their changes to the live mode they have
to interact with the publish button in
synapse studio so this is a manual step
in this workflow in today's video I will
show you how you can turn this step into
a automated step by using a new feature
in the synapse workspace deployment test
the validate and deploy operation type
so stay tuned here is a simple branching
strategy I'm working on two different
projects project a and Project B each
project will have their own feature
branches I have this the developer names
John rui working on different features
and then I will have an oddfixes branch
on my code on any release
that I'm developing in this releases
Branch so this is the the the branching
strategy that I have here for my
projects also the Gateway folders all
code that is ready to be deployed to uat
will land in these branches one for
project a and another one for Project B
to automate the the publish process we
want to avoid this publish button so I
don't want to merge my changes from my
project a
for example project a I have I'm working
on this feature and I will create a new notebook
notebook
just with this print my notebook
project a
selecting an Apache spark pool
and committing my changes now once my
development my development is finished
what I want to do is to merge my changes
bring this notebook to my Gateway Branch
for project a and when I merge these
changes what I want to do is to kick off
the deployment the publishing to a
different environment to this one
now how can we achieve that I want to
isolate this deployment so I don't want
to bring anything from Project B I just
want to deploy this notebook from
Project a into a uat environment let's
go to this option pipelines releases and
let's create a release pipeline I'm
going to create a new release pipeline I
will name I will start with an empty job
here I will name this as validate
validate
and deploy
multiple projects I will start by adding
an artifact Repose project synapse CI CD
select my repo and my default Branch can
be the Gateway
and I will call this my deaf Alias here
this is my artifact
I am going to set up this stage here and
this will be project a uat
uat
ready stage and I'm going to set up my
release this containers deployment
trigger every time I get push occur in
my repo in my Gateway Branch I want this
release to be triggered I'm going to
click on the add button to have the
branch filter I want this release to be
triggered every time a commit is done on
the Gateway Branch I will just type here
Gateway slash star in the pre-deployment
conditions I'm going to create this rule here
here
for this stage to kick off when that
push happens in the project a branch now
I will set up my test and I will add the
select the validate and deploy when we
use this validate and deploy we are
talking about two stages one the
validate that will be responsible to
generate the arm templates in runtime
and the deploy that will grab these arm
templates and publish to the Target
workspace what I'm going to do is to
Define my artifact root folder my Dev
and I will set up the connection type
workspace Resource Group which is my RG
synapse and then my synapse cicd u80
I think seems to be all good now I'm
going to save this and back to the
pipeline what I'm going to do is to clone
clone
this release I want this to be created
after the release name this as project
B uat ready the condition the
pre-deployment condition for this stage
is to kick off
when something occurs in the Gateway
Project B and foreign
let's do this
what I'm going to do here is to create a
pull request and merge my changes not
into main I don't want to touch my
collaboration Branch I just want to
publish my changes to a Target
environment but without changing
anything in the main branch the things
that have not been approved yet I want
them to be approved in a different
environment for testing and then once
they are approved then I can merge these
changes into my collaboration branch
what I want to do is to merge this into
my Gateway for project a this is a
project a feature I want this to be
automatically published to the uat
environment I will keep this feature a
branch I'm merging me my changes into my
project a
and if we go to the releases we will see
now this release running it's been
triggered now if we click on this
release you will see that this is now in
progress and the project B uat has not
been deployed because the artifact
conditions were not met now what I'm
going to do while this stage is in
progress I'm going back to my to synapse
Studio my feature Mark is working on a
new feature let's create a new SQL
script for Mark
okay select edit version for example
doesn't matter
Commit This is ready
to be deployed in uat create a pull
request and Mark's new feature will be merged
merged
into the project B Gateway Branch I will
complete
complete merge
and if we go now
to the
validate and deploy
a second release is now underway let's
click on the release tube and it will be
now the project B that will kick off
this stage
the project a uat ready stage the
conditions were not met and by doing
this we can automate the deployment so
no interaction with this publish button
button
from any user Branch without any
interaction with synapse studio if I go
now here to my uat let me just refresh
the browser here it is my notebook deployed
deployed
from Project a
and my SQL script it's been deployed
so now we have two artifacts from
different projects deployed to my target
workspace without the need to interact
with synapse Studio to publish these
changes now again if we go here to the
the validate and deploy
will generate the RM templates
and you can see here the arm templates
being generated here it is
the template for workspace
was generated to this folder exported
artifacts folder and the template
parameters for workspace as well now
these templates were generated using the
standard or default
parameters template if you want to use a
custom parameters template then what you
need to do is on the project a for
example in the root folder you must have
the template parameters definition Json
file the root folder of your branch
should have that definition file for the
arm template parameters to be generated
according to that definition in this
root folder that you set up here if you
go to edit pipeline Dev
here
you should have the template definition
Json file if not the default parameters
template will be used
now this is one scenario where you can
automate the publishing operation to a
Target environment of course that in
case you wanted to publish anything in
the same workspace the idea would be the
same but the target workspace of course
so here you should have your Dev workspace
workspace
in scenarios where you don't want to
publish your changes
to or merge your changes to the
collaboration branch and to publish the
changes directly to live mode so I just
want for example I have a pipeline I
don't want to merge my Pipeline with my
collaboration branch but I want to
trigger the pipeline so I need to publish
publish
so you can use this validate and deploy
feature to publish your pipeline from
your feature Branch to live mode and
there is no visibility on the main
branch that this pipeline exists so this
is another scenario where the validate
and deploy operation type can be helpful
let us know your thoughts and share your
ideas tips and tricks in the comments
below and don't forget to like comment
and subscribe our Channel
thanks for watching and I'll catch you
on the next one [Music]
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
