YouTube Transcript:
Building Operable Software with TDD (but not the way you think) - Martin Thwaites - NDC London 2023
Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.
Share:
Video Transcript
um this talk just in case you're in the
wrong room we're talking about building
operable software with tdd
um and the obviously a hook of not the
way you think so hopefully there'll be
some surprises in there
okay so my name is Martin Thwaites I go
by martin.net in everywhere
um I am first and foremost an
observability evangelist I'm also a
developer advocate for a company called honeycomb
honeycomb
um socials are there we have a booth
just outside the door come see us blah
blah blah they paid me to be here so
come see us
um awesome
so what we're going to talk about today
we're going to talk about
operable software
um we're going to talk about what my
definition of operable software is more
specifically because that's all that
matters we're going to talk about how we
test software as developers we're going
to look at what's something called
outside in testing
um what that looks like and we're going
to talk about how we can use
observability and how does that fit in
to that operable software life cycle
now before we get to that I want to talk
a little bit about the motivations for
this talk
before I started working with honeycomb
I spent a couple of years working for a
UK Bank building a new banking software
and one of the things about banking
software is it needs to be accurate and
correct apparently giving somebody the
wrong balance and approving things when
they shouldn't be approved is a bad thing
thing
um so basically being able to say that
my code is correct
on the bottom layer isn't good enough
because a system is bigger than your class
class
and we had to spend a lot of time
thinking about how we test software we
wanted to go full CI CD we wanted to
have really continuous release cycles
and a lot of that goes against the idea
of resilient software the idea of
software that doesn't change you know
banking industry is we'll do three
releases every Century
um and that was a bad thing in agile
software in building software that is
resistant to change all that kind of
stuff it just isn't a thing in banking
so we spent a lot of time building out
approaches and that kind of stuff so
what I'm going to talk about today is
not theoretical
it is something that we spent two years
building and is running in a production
environment well production-like
environment they're not actually taking
payments yet but it is running an
environment and it is running cycles and
it is something that works so hopefully I've
I've
I'm not going to say dumbed this down
but what I've done is I've tried to
simplify this into some of the component parts
parts
but do take away from this that it is
something that works
okay so what do we mean by operable
software or more specifically what do I
mean by operable software first and
foremost it's software that's built for production
production
now I hate to say this
I don't care whether it works on your machine
machine
it's not where customers are going to be
using it when we build software we build
it to be run in production that's the
thing that we're thinking about we're
not thinking about how we can run an
emulator locally and it doesn't matter
what matters is does it run in the
production environment when customers
are using it
so what does that mean
built to run in production it's built to
run on production servers it's built to
run in a production environment it's
built to run with multiple regions it's
built those are things that you're
it's also built with live operations in mind
mind
now some of you may come from the
environments in the past where you'd
write your code you'd throw it over and
somebody else would then have to deal
with it because that's not your job they
do with it unfortunately everybody's Ops
now everybody is Ops
um we are working in a devops culture
where developers and operations teams
come together
you need to build your software with the
people who are going to run it in mind
sometimes that's yourself
because
past Martin is let's be honest a little
bit of an because he put a lot
of problems my way and future mine we
can leave this off of him because you
know he's fine um
um
okay now the other thing is built for
fast recoverability there are things
called the Dora metrics which Google
built on the devops reports
um which it mentions something called
mttr it's falling a little bit out of
favor right now but it doesn't mean that
it's not something you should be working
to which is the idea of how do I build
things that recover fast from failures
and all of that leads to the idea that
my application needs to emit robust
Telemetry signals whether that's metrics
or traces or God forbid using logs but
that is something that you need to be
thinking about because in production you
can't attach a debugger has anybody
tried to attach a bugger a debugger to a
live running IIs instance
yeah did it go well
yes for those that don't know
um whenever you attach a debugger to a
live running process it stops all the
threads every request just holds it's
not a pleasant day okay
so when we talk about operable software
what do we mean by software
now the traditional definition is the
code that you write that does the thing
that the business wanted unfortunately
that isn't the case it's everything
it's the application you write it's the
test that are part of your code
it's the pipelines that you've built
that test
do security analysis and deploy your
code into production
it's also the infrastructure that's been
built for it that might not be you who's
built that infrastructure but it's still
part of the software
because the production environment is
part of your team that delivers value
into your customers
that is part of it
you can't host your application without infrastructure
infrastructure
I mean that's just shipping your own
machine and that never works so but the
other thing is it's your observability
tooling it's the live operations team
that need to know whether your system is
up whether it's functioning all of that
is part of the software that you're building
building
so when you're building software if
you're not considering these things
you're missing out
you're neglecting part of your software okay
okay
so let's talk a little bit about testing
let's talk a little bit specifically about
about tdd
so what do we mean by tdd um
um
tdd has been completed over the years I
mean it's it's very abnormal we don't
take terms in I.T and use them for
different things ever do we no that's
not we don't take a well-known term like
you know I don't know devops and distort
it into another thing
tdd was a very very basic concept
contract First Development develop your
contract develop your tests do the thing
it's also Red Green refactor
is the idea that you write a test that
fails you make the test pass you can
refactor your code and then it fails and
then there's a big cycle that goes through
through
but the main basis of tdd
is you write tests first and then you
write the implementation unfortunately
unfortunately
tdd has not meant that to a lot of
people and one of the things that we
found while writing this approach
was people didn't like tdd they said
they thought it didn't work it's it
takes too long
and when you dig into it it's because
what they're doing
is they're mocking all the things um
um
the.net developers among you will get
the reference there um
um
but they will mock everything
everything
they will have a class with 17
dependencies all of them with mock
and they will test individual methods
I've got a method that I've mocked out
all the dependencies for I will test
that method to death
now nobody cares about your 4 000 unit
tests I know you like them
I know you spent a lot of time on them
but nobody cares about the level of your
unit tests what they care about is your
so what does why do we do tdd
because if it has so many constraints
and it's so bad why is it that we're
doing it what are the benefits that
we're trying to get out of tdd
now tdd is known to improve the design
of contracts I put methods on there
might not be methods it might be API
contracts for your SDK it might be API
contracts for your HD http apis
but it's based on improving
the design of your method contracts
because you write a test and it passes
that test you should do the smallest
thing that makes it past that test which
means your interfaces evolve your
classes evolve
it also helps under you improve your
understanding of the code base because
you're using these methods
you're not writing a test
that does a very very small thing you're
writing something that's actually using
these apis in the way that other people
it also documents your code if you're
writing these tests well
your tests are going to actually be
documenting your code as you go
because you've wrote a test for a
business outcome or a particular class
that you wanted to do something if
you've wrote your test properly you've
actually got documentation already
which means you never need to go into
Confluence again
okay but what's the Bad and the bad here
comes from what I believe is the the way
that a lot of people run tdd in their teams
teams so
so
it's normally written these tests are
normally written in the absence of any
consumers of this application
you'll get a business requirement that
says let's do a transaction and what
you'll do is you'll write a transaction
class and then you'll write some methods
on your transaction class like add amount
amount
that's not what your consumers have
asked for
what you're doing is you're assuming
what that class needs to do and then
assuming what you're going to use that
in and then assuming assuming assuming okay
okay
and because of that
all of those tests become quite brittle
to change
I've added a new parameter I've changed
the parameter
well I'll just delete all those tests
um
that's bad because now all of a sudden
you don't know whether your system still
works because you've deleted all the
tests and stats on new ones which means
that you then end up with manual testing
from the outside in and all of that kind
of stuff okay
okay
let's talk a little bit about component
testing everybody's seen this meme I assume
assume
where what we end up with is lots and
lots of unit tests four thousand unit
tests on a single class and it's
bulletproof that class
absolutely that draw was probably the
smoothest opening draw that you've ever seen
seen
um
but that's where integration tests come
in now integration tests over the many
many years like with all the other
things we've been talking about have
been misconstrued and changed and they
mean 17 different things to 17 different people
people
um so what I like to call these that I'm
talking about is component tests
um they all fall under a banner of
something called developer tests which
is something that Ian Cooper's been
pushing for quite a while which is the
idea that these are tests that the
developers write
to ensure the completeness of their code
the completeness being correct Acura
does the right thing these are tests
that you write
so outside in testing is part of what I
consider the component testing workflow
and what that means is we're working
from the outside of our application
we're working from where consumers live
that could be your HTTP API it could be
your SDK that you've built but where are
your consumers the thing that you
deliver as a package as a thing how do
your consumers use it so we're working
right from the outside all the way to
the inside
now what what I found is the majority of
time you can actually stay 100 on the
outside which is great because you don't
need to be on the internals
what we're doing is we're testing the
inputs and outputs and I'm obviously
using the HTTP vernacular here because
I'm guessing that I would say over 50
here write web applications whether it's
apis all that kind of stuff
but that could be just using your apis
what it means is we're testing from our
consumers or users we're testing what
our users will do
they will add something via a post they
will go and get something fire get well
let's do the post and do the get then
but we're also testing specifically what
the businesses are asked us for our
users to be able to do
which means that we're really close to
only delivering what we need to do
now you might have delivered this this
class with 4 000 unit tests that has 15
methods on it and those methods are a
um and then you look at the actual usage
in your application there's only one of
them used and they use one property from
that object you've done too much
running from the outside in means that
you're only doing the things that you
need to actually build
so benefits
benefits
business requirements first
all of a sudden you've built you've
designed all your business requirements
you've satisfied all your business
requirements there's no need for that
further layer of testing
if you've got tests that prove each
individual step of your application
you end up with better readability
because you're not using your internal
business knowledge
you're not using a test that says
transaction service returns approved
approved
now what you're saying is the
transaction was approved
the transaction was cleared
authorization was failed
you're using the business terms inside
of your test which means that they are
more readable use English language and
we'll go through how we can actually
write some of these tests using some
but we're also testing the whole
we're testing our serialization we're
testing the way that our startup works
all of that stuff is being tested when
one of my favorites no stubs or mocks we
don't need them because we're testing
my absolute favorite though you get 100
co-co coverage I know it's a vanity
metric I do
but what's great about testing from the
outside in and testing for your business
requirements is if that code isn't
covered by any tests you can delete it
because it's not used
if it if it is supposed to be used guess
what you're missing a test and you need
to add a test
so you get 100 code coverage it's awesome
okay
a key part around using this approach
for me and the teams that I've worked
now what do we mean by readable tests I
mean test names that are descriptive
your compiler does not care how long
your method name is
it really doesn't
put actual English in there
tell me what that test is actually doing
so that when I'm looking through those
tests I can actually look at the test
name and say yeah that's what it's doing
however you do that
there's no hard and fast rule I'm not
going to say you need to segment it into
three sections with an underscore
generally how I do it but that doesn't
matter as long as I read this about the
method name
and it makes sense to me let's do that
we build things that are reusable we
abstract context into reusable methods
in our tests
that's something that I see every single
person miss that I work with
if you're writing tests you don't have
to write a test that's 72 lines long and
does everything
against the raw objects you don't need
to do that abstract that away give it
context give it a name say that I'm
going to go and get this thing not go to
so we're abstracting away implementation
that doesn't matter to us in our tests
it matters that we do it but it doesn't
matter that somebody sees the 17
different lines of code that do that
particular thing maybe in one test it
might be useful but it's not useful in
all of them foreign
foreign
but we also use our domain language what
the business understands what you're
understanding from your business
requirements because if you think about
what the business is doing you can then
actually look at somebody goes it fails
when we do an authorization you don't
have to understand how an authorization
works in the system you can go through
the tests and say where are the
authorization tests here with the
authorization test oh and it fails under
this scenario well this test says that
that's a specific thing great I can get
right to the test I can write more tests
and then I can bit fix it
the other thing is more tests not more
asserts this is probably the most
controversial part of it
people like to add 17 different asserts
equivalent to should not be used ever
I'm sorry
you should be writing more tests because
then your test names are giving you the
context you've got specific tests that
are failing that give you that context
of why that thing failed
when you've got one test that has 17
asserts on the bottom of it guess what
it stops at the top one
yes you can make it work with lots of
money now but generally stops at the top
one and you don't get all the other ones
it'd be better if you saw all the tests
foreign and that's the elephant in the room
room
isn't this just right in bdd
yes it is
it's not writing cucumber tests it's not
right inspectful
but yes we are defining Behavior
and the behavior is what's driving our
design of our application
but we're not writing given when lens
we're not building out a spec flow
framework with steps that are only used once
what we're doing is the original
definition of what bdd was about
we're talking about the behavior of our
system and we're having that drive the
design and drive the development so yes
yes it's bdd but not that bdd
I have opinions about cucumber and spec
flow come around the booth afterwards we
can rant okay
okay
so we'll look at a bit of code
now this is using.net
the same approach works with majority of
languages that can run up an in-memory
web server for your application
in this one we're using something called
the web application Factory
it's built into.net and allows you to
spin up your entire asp.net core site
um and be able to hit the apis correctly
this is a test you might write
which it's pretty simple it adds us to do
do
and make sure we get a valid request back
back
so I send a valid to do item over
and I get an okay response pretty simple
but what I've done here is I've defined
here there's a business requirement I
need to add to this to our system great
okay what does a to-do look like well
it's got title and description great
I've wrote a test
that will actually compile
whereas if you write this against actual
typed code
it's not going to compile first time
because those things don't exist
and what we're doing is what we're doing
is asserting that we get an okay back
because that's what the business
requirement said I need to be able to
add it to do they've not said they need
to be able to get it one of my favorite
things about this approach is you can be
really really pedantic with your BAS
when they go I need to add a to-do it's
like great better to do
that actually proves you can add it to do
do
I don't need a database
don't need to store it
I've got an endpoint that allows them to
add it's great which means that your
Ba's start to get into this idea of
thinking about exactly what it is that
they need
and over time you build up that
relationship with your Bas where they
actually think about what it is that
so how do we make that a little bit better
better
well the first thing I talked about was
removing implementation for that test I
don't care that there's a web
application Factory
this is called evident testing
so everything in that test
should be relevant to the test
everything else should be abstracted away
away
so obviously what I've done here is in
the background I've created the web
application Factory and just made it
accessible under a variable called API
which means that my test now if I say
add to do with valid request well it
so that's already a little bit cleaner
because there's a lot of implementation
details that are not relevant so that's
an easier test to read
but we can do better
I'm sure I'm assuming that the majority
of people have heard of AAA testing
arrange actor sir
well let's spread this out a little bit
well we'll create a to-do item
that's our arrange and then we'll act so
we'll throw the to-do item
against the API and then we'll assert
so that's again making this just a
little bit more readable but we can go
even further
because for this test I don't need to know
know
what the parameters are of that to do item
item
what I care about is I've got valid to
do item
so I can abstract that away and give it
an actual context of a name
that says go and get me a valid to do item
item
because again the shape of that to-do
item isn't relevant to the test
which means that all I'm doing is don't
get a value to the item throw the value
to do item against that
that's now something reusable so
so now the Ba's come to me and said well
when I add the to-do I need a unique ID
for it and that it needs to give me a
valid ID back like great okay I've got
another requirement
but you can see those first two lines
are exactly the same
all I've done is I've got a different
test name which tells me exactly what it
is I'm trying to test
and I've got some different asserts
does better ways to do this obviously
you could then start to abstract some of
your asserts out of that make those
asserts a little bit more and context driven
driven
but what we're ending up with
is something that's just a little bit
more easy to read
a little bit more separated
so that we can start to act on
individual tests and we can just add
more tests
we got so many times people saying I
don't want to add more tests because
it's more work
you know I've got to copy 70 lines of
code well that's the wrong reason not to
write tests
what you want to do is make it easy to
write new tests what these approaches
give you is something really really easy
in a framework that somebody says
there's a bug in production
and what you've got is a Lego
set of different tests that they go oh
this is broken it's like right grab one
let's grab one oh yeah it's broken
oh yeah oh I can fix that great fixed
job done and now I've got a test for it
guess what you've done you've wrote a
test first
and it failed and then you fixed it
okay so
so
then we can go for persistent check so
now they've come back to us and said
well when I add the to do I need to be
able to get it back like great okay now
I need a database
um but up until this point I didn't
um like I say I love being pedantic
um so this is now right we need to add a
database to this
but again what we've done
is we've abstracted away context
now if you read what this test does it
should be self-evident
it says create a valid to do item
now you could say
add a valid to do item to the API
whatever makes context important to you
whatever makes that understandable to people
people
but we've gone and created a to-do in
the API
we've not gone to the database
we're not understanding the scheme of
the database and injecting something
into the database we're using the API here
here
we're essentially calling into the
previous tests
that create a valid to do item
and what that's going to do is give us a
valid to do item back
and then we're going to go and get it
from the API
now at no point here am I touching the
database I'm not reaching into the
bowels of my application and
understanding what columns exist in the
database I don't need to
I can run this against the API because
that's what my users are going to do if
you've got users that are going into
your database and inserting to do items
into your database you need to have a
real think about your applications and
your users
that isn't what they're doing so why is
that what you're doing in your tests
nobody cares when there's a bug in
production and you say well I've got a
test that inserts into the database and
make sure it comes back I just didn't
test that I post it through the API and
it comes back nobody's going to take
that as a valid excuse
okay
recap a little bit about what we've
tested there
was testing our Startup configuration
how many people have created a service
class in.net
and had their 4 000 unit tests against
their service class and then forgot to
add it to dependency injection before it
at least in Oslo they were about 50 of
truthful people
um but that's what happens people will
write these tests and their unit tests
pass great ship it and then it fails to
start in production and you're like you
need to surpassing
and what you've actually forgot to do is
just register that implementation
against your service collection
the other thing with testing is routing
we're testing that the API routes that
because that happens as well
you get people who forget to put a guide
identifier a good qualifier on some of
their names so the routing looks like it
should work but it doesn't because
we're also testing serialization and deserialization
deserialization
because when you push stuff into your
application via the API it's going to do
your Json serialization deserialization
now Pro tip on this one
is when you're using this approach
rewrite all of your classes in your test
assembly do not reuse the classes in
your main assembly
because what that gives you is if you
have to change the classes in your test
assembly because of something that's
breaking in your main assembly all of
your consumers are going to have to do
the same
and what you can actually do is put path
approval checks inside of your commits
that says if somebody's changed this
somebody else needs to look at it
so what you've actually got is contract
testing built in to your application
if somebody changes the serialization
method from camel case to Pascal case on
the Json objects all your tests are
but if you use the same serialization
classes then you've got a problem
we're also testing model validation
required parameters regexes
because we're going through the full
HTTP pipeline when we do it
we're testing things like database interaction
interaction
if you're using Entity framework which
this is an example that's up on GitHub
that you can go and have a look at it's
using Entity Framework
it's using SQL Lite don't use the
in-memory database because constraints
don't work
but if you're using sqlite you've got a
database that you can use
so you're actually testing that the
database interaction Works your foreign
we're testing selecting of columns from
the database properly
making sure that you're actually using
the right column names
in your tests
but we're also testing the customer apis
we're not testing our classes right down
below we're not testing those
we're testing our apis the things our
customers are going to use which means
we're testing the business requirements
which means that we're testing only what
needs to be tested
I don't care that your class returns an
object with 17 properties on it if I
only need one of them
and we're testing all of that
and like I say if you do that you end up
now
just before we move on to that one
one of the main objections that I get
for this is it takes too long to run the tests
tests
because I'm running up an application so
it's going to take too long um
um
normally that's people who haven't tried it
it
we have around two and a half thousand
tests in that framework that run in
eight seconds
so anybody who says this takes too long
has not tried it and done it properly it
takes time to build Frameworks around it yes
yes
it will work out the box a lot of the
time you can reuse the web application
from Factory you can use different
contexts for your database so that you
get clean tests yes it takes a little
bit of work it's not too slow though
if you're worried about your test taking
eight seconds I'd ask you how long you
deploy is taking
because I guarantee you it's not less
but what can't we test this way
you know I've sold you the dream I'm
sure you're all going to go away
tomorrow or Monday and do all of this
stuff which is great
but what can we test
so we can't test things like how the
back end changes but the front end doesn't
doesn't
to the user cashing being one of the big
examples of that if I'm doing a get
against my API how do I test that
caching works
because I get the same result no matter
what if I'm not getting the same result
then my Test's broken I should get the
we can't test things like internal State
changes like audit logs
things like that
we can't test performance
anybody who thinks they can test
performance locally
you're also not testing integration with
cloud services
so you're not testing integration with
Cosmos you're not into date the
integration with Dynamo things like that
you can do that with this approach
because you're working from the outside in
in
what you can do is you can switch out
the implementation with the live
implementation and rerun the tests
maybe if you've got branching structure
something like that you run that test on
your main branch when it's on when it's
actually merged in but you don't run it
on every feature Branch maybe or the
other way around
but because you're running from the
outside in it's literally a case of in
your service collection just replacing
your implementation for I don't know
the other thing you can't test is
configuration so you can't test that
your deployment pipelines have put the
right key Vault ID into the config so
and this is where we start to talk a
little bit about observability
so all those things yes you can't really
test them with the approach that we just
talked about
but we can augment it a little bit in
order to get some of those techniques in there
there
so I'm gonna have to talk a little bit
about what observability is
and this is the definition from our
co-founder who first coined the term
observability in the context of software
about six years ago
so it's about understanding and
debugging on no no knowns it's about the
ability to understand the inner state of
our system by asking questions from the outside
outside
that's the statement that I'm more
we're writing observability into our applications
applications
writing those Telemetry things so that
we can understand our internal State
when we're in production
if we're writing operable software in a
way that means that we can use it in
production well can we not do the same
thing with observability in our local environments
environments
and that's what I set out investigating
about a year ago
and we came up with this concept I hate
the fact that we use the term odd um
um
because what we actually turned it was
observability during development which
is just really confusing but the idea is
how do we use observability or more
specifically tracing I wanted to call it
tracing during development but
apparently tdd's taken and we're not
allowed to just augment names and use
them for something else um
um but
but
this whole Concepts don't call it odd
call it whatever you want but it's the
idea of how do we bring observability
into our applications and our local
development folks
observability is an output it's a side
effect of our application it's something
that our application does
so why aren't we testing it if it's
observability can detect things like parallelization
parallelization
you can know whether two things run in
parallel or run in series
that's important in some scenarios
I have a few anecdotes that I can share
offline but one that I can share online
is a friend of mine went into an organization
organization
and they had a problem that the startup
of their desktop application was taking
30 seconds
and he went in put some open Telemetry
and tracing data in there and said well
when you do this grpc call you're
initiating the connection for everyone
so it's initiating the connection 20 000
times it's like no no it opens the
connection once and then just sends all
the data over he's like it doesn't
that's not what it does it's like no no
it absolutely does and it's like but
the computer's telling me that it
doesn't and computers don't lie they
only tell you exactly what they're doing
um and then they went in and went oh oh
yeah it's not
um and turned 20 seconds into 200 milliseconds
milliseconds
um just by three lines of code of reuse
the connection
so observability can tell you these
things and they're important in a lot of applications
the other thing it can do is detect code
paths it can detect if you go through
say caching
if the caching code is called
you can tell if this was a cached
version or a not cache version
but the key is if it's important to you
locally for you to be able to write a
test on your class about it it's likely
going to be important when it gets onto
production and it goes wrong
it's likely going to be important that
you know that that's where it went wrong
in your application
what you really want is when somebody
sends you a book
what you want them to do is say it was
on this line here and it sent the wrong
information all right I'll change that
line there and commit it and push it
like that that's a really good QA who
can come to you and say it's this line
of code can you fix this line of code
observability done properly with tracing
causality all of those kind of things
can really show you internally in the
application what it does
so if that's important why aren't we
bringing that into our testing why
aren't we bringing that into our
so let's take a look at some caching code
for any future employers out there that
may look at hiring me I don't write
caching code like this this is an example
example
it's very basic I wouldn't write it like this
this
um so to give me an idea of what this is
doing for those that don't read c-sharp
which I'm hoping is a minority what
we're doing is we're setting a cache key
we're using a memory cache here to try
and get a list of items list of to-do
items using that cache key if that
didn't return anything then we're going
to go to the database and go and get the
to-do items and then we're going to
return the ones from the database instead
instead
the methodology there is pretty sound
that's how you do a lot of things in
applications you'd have sliding
expirations all that kind of stuff
but that's pretty much what a cash code
would do
now the thing is
both the line that says return to-do list
list
and the one that says return all to do's
they're both returning the same information
information
so external to the application I don't
know whether it's actually got the data
from the cache or I've got it from the database
so when I write my test
and write my test that says well the
first call needs to come from the database
database
I don't know how to write that because
whether it comes from the database or it
comes from the cache it's going to be
the same so how do I do it yes I could
write headers to it but those can be
messed up
and this is where something called open
Telemetry comes in
so open Telemetry is the number two
projects to come out of the cloud
compute native Cloud native compute Foundation
Foundation
behind this little small project they
did called kubernetes don't know whether
you've heard of it um
um
but it's the number two projects come
out there it's becoming the de facto
standard in How We Do Telemetry
signaling Telemetry signaling being at
it's completely open source
it's run by a lot of the vendors the
vendors have been building this as an
open source project
but one of the really interesting things
about it is something called the
in-memory exporter now what that means
is and especially in the.net world
your production application does not
need to depend on open telemetry
in order to get Telemetry out
because a lot of organizations will go
we're not we're not ready to take open
Telemetry into our production
application yeah there's some problems
but because
net uses something called activity
activity has been built into.net since
it's either done at 2 or 3.5 and it's
changed a little bit since but it's part
of the BCL
so you can instrument your code with activity
activity
without thinking about open telemetry
and then you can actually bring open
Telemetry into your test because largely
organizations don't care what Frameworks
you bring into tests they care about
what's deployed to production
and what we're doing here is we're
creating a tracer provider
inside of our web application Factory
not inside of our code
not touching the production code here
but in all we're doing here is saying
listen to this particular activity source
source
bringing the asp.net core
instrumentation which will give us spans
for requests
and any spans that are created any
activities that are emitted from your
application stick them in this list
so we've got a list now
got a list of individual spans that have
been created as part of our test
and then what we do
is we add it to our services
because when you're using the custom web
application Factory
you can add additional things to your
service collection that are only present
when your tests wrong
now because I've added this as a
signaltern like that
that means it's going to start listening
to all of the activities that are run
and created during my tests
so that's pretty interesting we can do
that another Pro tip when you're writing
your custom web application tests run
this clear providers
it'll save you a ton of time in your
tests because if you don't and you've
accidentally enabled console logging
your tests take a hell of a lot longer
because it outputs all of your logs to
the console which takes a lot more time
a bonus nothing to do with what we're
talking here
I don't charge extra for that just to be clear
clear
so what does our code look like now
well we wrap
our database call
inside of a span we get the added benefit
benefit
that we now get to know how long it
takes to get things from the database
that's something that's interesting in
our application
it's not something we're going to test
against because we can't test
performance locally it's pointless
because we're running against an
in-memory SQL and
SQL light will be a lot faster or slower
depending on what you're doing
we're going to wrap it in there and give
it a specific name
that's it
it's not particularly complicated but
now what we can do
is in our test
we can say
well find all of those collected spans
that have been created
and tell me did it contain one that had
get to do this from the DB well this
test is making sure that the first cold
calls the database
I actually wouldn't write this test
because that's pointless
because we know that the first test can
will only pass if it's got data
so kind of pointless test this one
the second test though is the inverse of that
that
well what we want to do is make sure
that the second cold does not touch the database
database
and what we can do there is we'll run
our first get
clear out all the spans
run our second get and make sure we
don't have
something called that
so now what we've got is a really
actually useful test that tells us
whether it's going to go to the cache or
not or is whether it's more specifically
whether it's going to call the database
or not to get that data
obviously very very simplistic test here
there's a lot of nuance that will be in
your application that you'll need to
throw into that
do you use I distributed cash and you
have to use the in-memory version of
distributed cache you have to make sure
those things are serializable
that's another thing that people miss
when they try and use I distributed cash
which needs serializable entities
whereas in-memory cache does not
and that fails when you put the actual
real reddish cache in place but it
okay
now the other ones that we did we
so now I can take that first test where
it calls the database
I can make sure well Collective spans
has span with name
I've got a really readable test
that tells me that I could even go
further and have a static list a
constants of all my spam names
because that might be important that
they remain consistent throughout
releases you can have them in shared
libraries for instance that you can use
and that could actually be referenced in
a constant there's lots of stuff that
you can do but the main thing is look at
your tests make sure they're readable
make sure they're testing the things
now the other thing I said is we could
talk about parallel processing we can
make sure that these things are run in parallel
parallel
that's something that you can test with
observability much much harder to test
when you're running actual code even
when you're running code inside of a
unit test where you're just test in a
class it's still hard to test parallelization
parallelization
but what we can do with this is say well
get me the spans with this name so start get
get
bad name but hey ho
and then get me all of the spans that
say get from DB so maybe in this example
what we've got is it's actually doing
seven or eight calls to the database but
we don't want to run them in series we
want to make sure that it runs all five
of them at the same time because that's quicker
quicker
and then we can say well make sure that
every single one of our processing spans
has a parent ID of our new span you
could do things like make sure that none
of the spans start after the other spans
there's lots of ways to do this
that allow you to be able to see into
your code
and see what it's doing
so a couple of do's and don'ts
use it for things you care about in production
use it to test things that are important
don't use it to test things that are not important
do make sure that you're thinking about
the data on your spans
that might be important to you take the
first two into account
but you can test against some decent
data in your spans
don't check the database was called
directly don't test your query don't
test that it hit the database with
select star from users
that's not what it's for that's abusing
the system don't abuse the system
don't check that an individual method
was called
don't create a span for every method and
then go in and say did it cull this
particular algorithm method that's not
important to your test that's how we
now the other thing that we can do once
we've got all of this in place
we're using open Telemetry in our tests
what we can then do is we can use
tracing tools to start to understand our
applications better
during our test Cycles
so what we can do is we can take all of
our test runs
and we can send them into something like
honeycomb it's not specific to Honeycomb
but you can
we can send all of those in and we can
say well how many spans were created
during that test how many spans were
created during the next test over that
whole run how long did the whole run
take what was the average time for the
test things that you used to get out of
the box
when you were using things like team
City and things like that when you move
into using say GitHub a lot of those
tools are lost
but you can use that now the other thing is
is
that because we're producing traces
inside of our application now
and we're using open telemetry we can
actually see the traces and visualize
the traces in your Trace tool
so this is an example of that does not
call database test
what I can do is I can look at that and
say well what is the shape of the trace
for that particular thing when I ran the test
test
now if you're using these
when you go to production
if you use the same tools
in production for tracing your
production application as you are using locally
locally
you can start to see what that should
look like in production and when you
look at it and go well no no no that's
that's not how it should work
you're getting more knowledge about how
so
some some interesting reading on stuff
the repository is there Martin JT to do odd
odd
a couple of interesting other talks to
follow on
tdd Revisited is a talk that in Cooper
did for the first virtual NDC back in 2021
2021
um which a lot of the concepts that I've
come up with about how we decompose tdd
into something that's useful come from
talking to Ian and that talk it's a
really really interesting talk about
what tdd is and is not
the other one is David Whitney talking
about writing tests that don't suck
which is a really interesting talk about
the refactoring and naming stuff that
we're talking about and extracting
context run from a JavaScript context
but it's exactly the same context me and
David worked on this for the last two
years so all of those concepts are very
very the same um
um
okay sales spiel
we have a free forever account with
honeycomb so the things that we were
talking about where you put open
Telemetry in the end of your application
you can actually send all of that to
Honeycomb for free for yourself locally
if you want to
we allow 20 million events to come
through US per month for free forever
if you're writing more than 20 million
spans in your tests per month let's talk
because I'd like to know what you're doing
doing
um we also have we wrote the book on
observability engineering it's free to
download which you can get on the
website which is not honeycomb at all
this is all about how observability
Works what observability is and there is
a full chapter on odd in there
but yeah that's me I'm happy to take questions
questions
um we have a ton of stickers
um we had a thing where people said oh
we'll do a sticker exchange and see
whose stickers are best and my response
was what's it like to lose
um all the best stickers are on our
booth out there come and take some
stickers or come talk to me you don't
have to talk to me just take the
stickers that's fine
um but yeah I'll take questions now but
come by the booth come say hello if you
want to demo if honeycomb well happy to
do it [Applause]
oh and also please do do the ratings
outside it really helps me if there's
specific feedback of stuff you wanted to
see on and didn't that kind of stuff do
tell me ping me on Twitter it all helps
other people who are seeing these
presentations to make them better do it
for me do it for all the other speakers
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
Works with YouTube, Coursera, Udemy and more educational platforms
Get Instant Transcripts: Just Edit the Domain in Your Address Bar!
YouTube
←
→
↻
https://www.youtube.com/watch?v=UF8uR6Z6KLc
YoutubeToText
←
→
↻
https://youtubetotext.net/watch?v=UF8uR6Z6KLc