0:00 [Music] foreign
0:12 and I'm a supporters Coalition engineer
0:14 on the Azure synapse analytics team
0:17 welcome to this video series on synapse
0:18 CI CD
0:21 in today's video we will be covering
0:24 some new and exciting features recently
0:27 delivered by synapsia City team to do
0:30 synapse workspace deployment extension
0:33 this new features will allow you to
0:36 automate the publishing of your
0:39 artifacts from any user branch
0:42 as you know until recently the arm
0:44 template generation could only be
0:47 triggered from synapse Studio when using
0:49 the publish button
0:53 now with these new features you can pick
0:55 any user Branch to generate These Arm
0:58 templates and how to make this
1:01 publishing operation from either your
1:04 Azure devops automated workflow or GitHub
1:05 GitHub
1:08 let's watch the demo and see how you can
1:09 achieve this
1:13 consider this typical CI CD workflow
1:15 where you have multiple developers
1:18 working in feature branches and when
1:21 they need to merge their changes to the
1:23 master Branch they create a pull request
1:26 all changes are now part of the
1:29 collaboration branch and if they want to
1:32 generate the arm templates and publish
1:34 their changes to the live mode they have
1:36 to interact with the publish button in
1:39 synapse studio so this is a manual step
1:43 in this workflow in today's video I will
1:46 show you how you can turn this step into
1:50 a automated step by using a new feature
1:52 in the synapse workspace deployment test
1:56 the validate and deploy operation type
1:59 so stay tuned here is a simple branching
2:01 strategy I'm working on two different
2:05 projects project a and Project B each
2:07 project will have their own feature
2:10 branches I have this the developer names
2:13 John rui working on different features
2:17 and then I will have an oddfixes branch
2:19 on my code on any release
2:22 that I'm developing in this releases
2:26 Branch so this is the the the branching
2:29 strategy that I have here for my
2:33 projects also the Gateway folders all
2:36 code that is ready to be deployed to uat
2:39 will land in these branches one for
2:42 project a and another one for Project B
2:45 to automate the the publish process we
2:49 want to avoid this publish button so I
2:51 don't want to merge my changes from my
2:52 project a
2:56 for example project a I have I'm working
2:59 on this feature and I will create a new notebook
3:01 notebook
3:06 just with this print my notebook
3:08 project a
3:10 selecting an Apache spark pool
3:14 and committing my changes now once my
3:16 development my development is finished
3:21 what I want to do is to merge my changes
3:24 bring this notebook to my Gateway Branch
3:27 for project a and when I merge these
3:29 changes what I want to do is to kick off
3:33 the deployment the publishing to a
3:35 different environment to this one
3:38 now how can we achieve that I want to
3:40 isolate this deployment so I don't want
3:43 to bring anything from Project B I just
3:45 want to deploy this notebook from
3:48 Project a into a uat environment let's
3:52 go to this option pipelines releases and
3:55 let's create a release pipeline I'm
3:58 going to create a new release pipeline I
4:00 will name I will start with an empty job
4:03 here I will name this as validate
4:05 validate
4:07 and deploy
4:12 multiple projects I will start by adding
4:17 an artifact Repose project synapse CI CD
4:21 select my repo and my default Branch can
4:23 be the Gateway
4:27 and I will call this my deaf Alias here
4:30 this is my artifact
4:34 I am going to set up this stage here and
4:37 this will be project a uat
4:39 uat
4:43 ready stage and I'm going to set up my
4:45 release this containers deployment
4:49 trigger every time I get push occur in
4:53 my repo in my Gateway Branch I want this
4:55 release to be triggered I'm going to
4:57 click on the add button to have the
5:00 branch filter I want this release to be
5:03 triggered every time a commit is done on
5:08 the Gateway Branch I will just type here
5:12 Gateway slash star in the pre-deployment
5:15 conditions I'm going to create this rule here
5:17 here
5:20 for this stage to kick off when that
5:25 push happens in the project a branch now
5:28 I will set up my test and I will add the
5:35 select the validate and deploy when we
5:37 use this validate and deploy we are
5:40 talking about two stages one the
5:42 validate that will be responsible to
5:45 generate the arm templates in runtime
5:48 and the deploy that will grab these arm
5:51 templates and publish to the Target
5:53 workspace what I'm going to do is to
5:57 Define my artifact root folder my Dev
6:00 and I will set up the connection type
6:04 workspace Resource Group which is my RG
6:10 synapse and then my synapse cicd u80
6:13 I think seems to be all good now I'm
6:15 going to save this and back to the
6:18 pipeline what I'm going to do is to clone
6:19 clone
6:22 this release I want this to be created
6:28 after the release name this as project
6:32 B uat ready the condition the
6:35 pre-deployment condition for this stage
6:37 is to kick off
6:40 when something occurs in the Gateway
6:42 Project B and foreign
6:48 let's do this
6:52 what I'm going to do here is to create a
6:55 pull request and merge my changes not
6:57 into main I don't want to touch my
7:01 collaboration Branch I just want to
7:03 publish my changes to a Target
7:05 environment but without changing
7:07 anything in the main branch the things
7:09 that have not been approved yet I want
7:11 them to be approved in a different
7:13 environment for testing and then once
7:16 they are approved then I can merge these
7:19 changes into my collaboration branch
7:23 what I want to do is to merge this into
7:26 my Gateway for project a this is a
7:29 project a feature I want this to be
7:32 automatically published to the uat
7:35 environment I will keep this feature a
7:39 branch I'm merging me my changes into my
7:41 project a
7:44 and if we go to the releases we will see
7:46 now this release running it's been
7:49 triggered now if we click on this
7:51 release you will see that this is now in
7:55 progress and the project B uat has not
7:57 been deployed because the artifact
8:00 conditions were not met now what I'm
8:02 going to do while this stage is in
8:06 progress I'm going back to my to synapse
8:09 Studio my feature Mark is working on a
8:12 new feature let's create a new SQL
8:14 script for Mark
8:18 okay select edit version for example
8:19 doesn't matter
8:22 Commit This is ready
8:25 to be deployed in uat create a pull
8:28 request and Mark's new feature will be merged
8:29 merged
8:33 into the project B Gateway Branch I will
8:38 complete
8:40 complete merge
8:43 and if we go now
8:45 to the
8:48 validate and deploy
8:50 a second release is now underway let's
8:54 click on the release tube and it will be
8:57 now the project B that will kick off
8:59 this stage
9:02 the project a uat ready stage the
9:04 conditions were not met and by doing
9:07 this we can automate the deployment so
9:10 no interaction with this publish button
9:11 button
9:21 from any user Branch without any
9:23 interaction with synapse studio if I go
9:27 now here to my uat let me just refresh
9:30 the browser here it is my notebook deployed
9:32 deployed
9:33 from Project a
9:39 and my SQL script it's been deployed
9:42 so now we have two artifacts from
9:45 different projects deployed to my target
9:48 workspace without the need to interact
9:50 with synapse Studio to publish these
9:54 changes now again if we go here to the
10:01 the validate and deploy
10:04 will generate the RM templates
10:06 and you can see here the arm templates
10:11 being generated here it is
10:13 the template for workspace
10:17 was generated to this folder exported
10:20 artifacts folder and the template
10:22 parameters for workspace as well now
10:25 these templates were generated using the
10:27 standard or default
10:30 parameters template if you want to use a
10:33 custom parameters template then what you
10:38 need to do is on the project a for
10:40 example in the root folder you must have
10:43 the template parameters definition Json
10:47 file the root folder of your branch
10:51 should have that definition file for the
10:53 arm template parameters to be generated
10:55 according to that definition in this
10:58 root folder that you set up here if you
11:01 go to edit pipeline Dev
11:07 here
11:09 you should have the template definition
11:13 Json file if not the default parameters
11:16 template will be used
11:19 now this is one scenario where you can
11:21 automate the publishing operation to a
11:24 Target environment of course that in
11:28 case you wanted to publish anything in
11:31 the same workspace the idea would be the
11:33 same but the target workspace of course
11:41 so here you should have your Dev workspace
11:43 workspace
11:46 in scenarios where you don't want to
11:48 publish your changes
11:50 to or merge your changes to the
11:52 collaboration branch and to publish the
11:54 changes directly to live mode so I just
11:57 want for example I have a pipeline I
11:59 don't want to merge my Pipeline with my
12:01 collaboration branch but I want to
12:03 trigger the pipeline so I need to publish
12:04 publish
12:06 so you can use this validate and deploy
12:09 feature to publish your pipeline from
12:11 your feature Branch to live mode and
12:13 there is no visibility on the main
12:17 branch that this pipeline exists so this
12:19 is another scenario where the validate
12:23 and deploy operation type can be helpful
12:25 let us know your thoughts and share your
12:28 ideas tips and tricks in the comments
12:32 below and don't forget to like comment
12:34 and subscribe our Channel
12:37 thanks for watching and I'll catch you
12:38 on the next one [Music]
