Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: This episode is sponsored by our friends at Fortinet. Fortinet’s partnership with AWS is a better-together combination that ensures your workloads on AWS are protected by best-in-class security solutions powered by comprehensive threat intelligence and more than 20 years of cybersecurity experience. Integrations with key AWS services simplify security management, ensure full visibility across environments, and provide broad protection across your workloads and applications. Visit them at AWS re:Inforce to see the latest trends in cybersecurity on July 25-26 at the Boston Convention Center. Just go over to the Fortinet booth and tell them Corey Quinn sent you and watch for the flinch. My thanks again to my friends at Fortinet.

Corey: Let’s face it, on-call firefighting at 2am is stressful! So there’s good news and there’s bad news. The bad news is that you probably can’t prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.

Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. I’m a cloud economist at The Duckbill Group, and that was a fun thing for me to become because when you’re starting to set out to solve a problem, well, what do you call yourself? I find that if you create a job title for yourself, well, no one knows quite how to categorize you and it leads to really interesting outcomes as a result. My guest today did something very similar. Maish Saidel-Keesing is an EntReloper, or Enterprise Developer Advocate, specifically for container services at AWS. Maish, thank you for joining me.

Maish: Thank you for having me on the show, Corey. It’s great to be here.

Corey: So, how did you wind up taking a whole bunch of words such as enterprise, developer, advocate because I feel like the way you really express seniority at big companies, almost as a display of dominance, is to have additional words in your job title, which all those words are very enterprise-y, very business-y, and very serious. And in container services to boot, which is a somewhat interesting culture, just looking at the enterprise adoption of the pattern. And then at AWS, whose entire sense of humor can be distilled down into, “That’s not funny.” You have the flexibility to refer to yourself as an EntReloper in public. I love it. Is it just something you started doing? Was there, like, 18 forms of approval you had to go through to do it? How did this happen? I love it.

Maish: So, no. I didn’t have to go through approval, of course. Same way, you didn’t call yourself a cloud economist with anybody else’s approval. But I got the idea mostly from you because I love your term of coining everybody who’s in developer advocacy or developer relations as a DevReloper. And specifically, the reason that I coined the term of an EntReloper—and actually looked it up on Google to see if anybody had actually used that term before, and no they haven’t—it’s the fact of I came into the position on the premise of trying to bring the enterprise voice of the customer into developer advocacy.

When we speak about developer advocates today, most of them are the people who are the small startups, developers who write the code, and we kind of forget that there is a whole big world outside of, besides small startups, which are these big, massive, behemoth sort of enterprise companies who kind of do things differently because they’ve been around for many, many years; they have many, many silos inside their organizations. And it’s not the most simple thing to open up your laptop, and install whatever software you want on, because some of these people don’t even give you admin rights on your laptop, or you’re allowed to ssh out to a computer in the cloud because also the same thing: everything is blocked by corporate firewalls where you have to put in a ticket in order to get access to the outside world. I worked in companies like that when I was—before I moved to Amazon. So, I want to bring that perspective to the table on behalf of our customers.

Corey: Bias is a very funny thing. I spent the overwhelming majority of my career in small environments like you describe. To me a big company is one that has 200 people there, and it turns out that there’s a whole ‘nother sense of scale that goes beyond that. And there’s, like, 18 different tiers beyond. But I still bias based upon my own experiences when I talk about how I do things and how I think about things to a certain persona that closely resembles my own experiences where, “Just install this thing as a tool and it’ll be great,” ignoring entirely, the very realistic fact that you’ve got an entire universe out there of people who are not empowered to install things on their own laptop, for example.

How is developer advocacy different within enterprises than in the common case of, “We’re a startup. We’re going to change the world with our amazing SaaS.” Great, maybe you will. Statistically, you won’t. But enterprises have different concerns, different challenges, and absolutely a different sense of scale. How is the practice of advocacy different in those environments?

Maish: So, I think the fact is, mostly working on standardization from the get-go that these big enterprises want things to work in a standard way where they can control it, they can monitor it, they can log everything, they can secure it mostly, of course, the most important thing. But it’s also the fact that as a developer advocate, you don’t always talk to developers within the enterprise. You also have to talk to the security team and to the network team and to the business itself or the C-level to understand. And as you also probably have found out as well in your job, you connect the people with inside the business one to another, these different groups, and get them talking to each other to make these decisions together. So, we act as kind of a bridge in between the people with inside their own company where they don’t really talk to each other, or don’t have the right connections, or the right conduit in order for them to start that conversation and make things better for themselves.

Corey: On some level, my line about developer relations, developer advocacy, has generally taken the track of, “What does that mean? Well, it means you work in marketing, but they’re scared to tell you that.” Do you view what you do as being within marketing, aligned with marketing, subtly different and I’m completely wrong, et cetera, et cetera? All positions are legitimate, by the way.

Maish: So, I think, at the position that I’m currently in, which is a developer advocate but for the service team, is slightly different than a marketing developer advocate. The marketing developer advocate—and we have many of them which are amazing people and doing amazing work within AWS—their job is to teach everybody about the services and the capabilities available within AWS. That is also part of my job, but I would think that is the 40% of my job. I also go on stage, I go on podcasts like this, I present at conferences, I write blog posts. I also do the kind of marketing work as well.

But the other 60% of my job as a service developer advocate is to seek out the feedback, or the signals, or the sentiment from our customers, and bring that back into the service teams, into the product management, into the engineering teams. And, as I said, sit as the enterprise customer in the chair in those meetings, to voice their concerns… their opinions, how they would like the products to go, how we can make the products better. So, the 60% is mostly what we call inbound, which is taking feedback from our customers back into the service teams directly in order to have some influence on the roadmap. And 40% is the outbound work, which we do, as I said, conferences, blog posts, and things like this.

Corey: I have a perception. And I am thrilled to be corrected on this because it’s not backed by data; it’s backed by my own biases—and some people tend to conflate the two; I strive not to—that there’s a—I think the term that I heard bandied around at one point was ‘the dark matter developers.’ These are folks that primarily work in .NET or Java. They work for companies that are not themselves tech companies, but rather tech is a supporting function, usually in a central IT-style organization, that supports what the business actually does, and they generally are not visible to a lot of traditional developer advocacy approaches.

They, by and large, don’t go to conferences, they don’t go on Twitter to yell at people about things, they commit the terrible sin—according to many startup folks—of daring to view the craft of writing software as this artistic thing, and they just view it as a job and a thing to make money for—filthy casuals—as opposed to this higher calling that’s changing the world. Which I think is wild take. But there are a tremendous number of people out there who do fit the profile of they show up, they do their jobs working on this stuff, they don’t go to conferences, they don’t go out into the community, and they just do their job and go home. The end. Is that an accurate perception? Are there large swaths of folks like that in the industry, and if so, do they centralize or congregate more around enterprises than they do around smaller companies?

Maish: I think that your perception is correct. Specifically, for my experience, when I worked, for example, my first two years before I was a developer advocate, I was an enterprise solutions architect which I worked with financial institutions, which are banks, which usually have software which are older than me, which are written in languages, which are older than I am. So, there are people which, as they say, they come there to—they do their job. They’re not interested in looking at Twitter, or writing blog posts, or participating in any kind of thing which is outgoing. And they just, they’re there to write the code. They go home at the end of the day.

They also usually don’t have pagers that page them in middle of the night because that’s what you have operations teams for, not developers because they’re completely different entities. So, I do think your perception might be correct, yes. There are people like that when you say, these dark matter people, dark matter developers.

Corey: And I don’t have any particular problem. I’m not here to cast shade on anything that they’re doing, to be very clear.

Maish: Not at all.

Corey: Everyone makes different choices and that’s great. I don’t think necessarily everyone should have a job that is all-consuming, that eats them alive. I wish I didn’t, some days. [laugh]. The challenge I have for you then is, as an EntReloper, how do you reach folks in positions like that? Or don’t you?

Maish: I think the way to reach those people is to firstly, expose them to technology, expose them to the capabilities that they can use in AWS in the cloud, specifically with my position in container services, and gain their trust because that’s one of the LPs in Amazon itself: customer obsession. And we work consistently in order to—with our customers to gain their trust and help them along their journey, whatever it may be. If it might be the fact, okay, I only want to write software for nine to five and go home and do everything afterwards, which most normal people do without having to worry about work, or they still want to continue working and adopt the full model of you build it, you own it; manage everything in production on their own and go into the new world of modern software, which many enterprises, unfortunately, are not all the way there yet, but hopefully, they will get there sooner than later.

Corey: There’s a misguided perception in many corners that you have to be able to reach everyone at all times; wherever they are, you have to be able to go there. I don’t think that’s true. I think that showing up and badgering people who are just trying to get a job done into, “Hey, have you heard the good word of cloud?” It’s like, evangelists knocking on your door at seven o’clock in the morning on a weekend and you’re trying to sleep in because the kids are somewhere else for the week. Yeah, I might be projecting a little bit on that.

I think that is the wrong direction to go. And I find that being able and willing to meet people where they are is key to success on this. I’m also a big believer in the idea that in any kind of developer advocacy role, regardless whether their targets are large, small, or in my case, patently ridiculous because my company is in fact ridiculous in some ways, you have to meet them where they are. There’s no choice around that. Do you find that there are very different concerns that you have to wind up addressing with your audience versus a more, “Mainstream,” quote-unquote, developer advocacy role?

Maish: For the enterprise audience, they need to, I would say, relate to what we’re talking to. For an example, I gave a talk a couple of weeks ago on the AWS Summit here in Tel Aviv, of how to use App Runner. So, instead of explaining to the audiences how you use the console, this is what it does, you can deploy here, this is how the deployments work, blue, green, et cetera, et cetera, I made up an imaginary company and told the story of how the three people in the startup of this company would start working using App Runner in order to make the thing more relatable, something which people can hopefully remember and understand, okay, this is something which I would do as a startup, or this is what my project, which I’m doing or starting to work on, something I can use. So, to answer your question, in two words, tell stories instead of demo products.

Corey: It feels like that’s a… heavy lift, in many cases, because I guess it’s also partially a perception issue on my part, where I’m looking at this across the board, where I see a company that has 5000 developers working there and, like, how do you wind up getting them to adopt cloud, or adopt new practices, or change anything? It feels like it’s a Herculean, impossible task. But in practice, I feel like you don’t try and do all of that at once. You start with small teams, you start with specific projects, and move on. Is that directionally accurate?

Maish: Completely accurate. There’s no way to move a huge mothership in one direction at one time. You have to do, as you say, start small, find the projects, which are going to bring value to the company or the business, and start small with those projects and those small teams, and continue that education within the organization and help the people with your teaching or introducing them to the cloud, to help others within inside their own organization. Make them, or enable them, or empower them to become leaders within their own organization. That’s what I tried to do, at least.

Corey: You and I have a somewhat similar background, which is weird given that we’ve just spent a fair bit of time talking about how different our upbringings were in tech at scales of companies and whatnot, but we’re alike in that we are both fairly crusty, old operations-side folks, sysadmins—

Maish: [laugh]. Yep.

Corey: —grumpy people.

Maish: Grumpy old sysadmins. Yeah, exactly.

Corey: Exactly. Because do you ever notice there’s never a happy one? Imagine that. And DevOps was always a meeting of the development and operations, meaning everyone’s unhappy. And there’s a school of thought that—like, I used to think that, “Oh, this is just what we call sysadmins once they want a better title and more money, but it’s still the same job.”

But then I started meeting a bunch of DevOps types who had come from the exact opposite of our background, where they were software developers and then they wound up having to learn not so much how the code stuff works the way that we did, but rather how systems work, how infrastructure works. Compare and contrast those for me. Who makes, I guess, the more successful DevOps engineer when you look at it through that lens?

Maish: So, I might be crucified for this on the social media from a number of people from the other side of the fence, but I have the firm belief that the people who make the best DevOps engineers—and I hate that term—but people who move DevOps initiatives or changes or transformations with organization is actually the operations people because they usually have a broader perspective of what is going on around them besides writing code. Too many times in my career, I’ve been burned by DNS, by a network cable, by a power outage, by somebody making a misconfiguration in the Puppet module, or whatever it might have been, somebody wrote it to deploy to 15,000 machines, whatever it may be. These are things where developers, at least my perception of what developers have been doing up until now, don’t really do that. In a previous organization I used to work for, the fact was, there was a very, very clear delineation about between the operations people, and the developers who wrote the software. We had very hard times getting them into rotations for on-call, we had very hard times educating them about the fact that not every single log line has to be written to the log because it doesn’t interest anybody.

But from developer perspective, of course, we need that log because we need to know what’s happening in the end. But there are 15, different thousand… turtles all the way down, which have implications about the number of log lines which are written into a piece of software. So, I am very much of the belief that the people that make the best DevOps engineers—if we can use that term still today—are actually people which come from an operations background because it’s easier to teach them how to write code or become a programmer than the other way round of teaching a developer how to become an operations person. So, the change or the move from one direction from operations to adding the additional toil of writing software is much easier to accomplish than the other way around, from a developer learning how to run infrastructure at scale.

Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don’t leave managing your database to your cloud vendor because they’re too busy launching another half-dozen managed databases to focus on any one of them that they didn’t build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.

Corey: I once believed much the same because—and it made sense coming from the background that I was in. Everyone intellectually knows that if you’re having trouble with a piece of equipment, have you made sure that it’s plugged in? Yes, everyone knows that intellectually. But there’s something about having worked on a thing for three hours that wasn’t working and only discover it wasn’t plugged in, that really sears that lesson into your bones. The most confidence-inspiring thing you can ever hear from someone an operations role is, “Oh, I’ve seen this problem before. Here’s how we fixed it.”

It feels like there are no junior DevOps engineers, for lack of a better term. And for a long time, I believed that the upcoming and operational side of the world were in fact, the better DevOps types. And in the fullness of time, I think a lot of that—at least my position on it—was rooted in some level of insecurity because I didn’t know how to write code and the thing that I saw happening was my job that I had done historically was eroding. Today, I don’t know that it’s possible to be in the operation space and not be at least basically conversant with how code works. There’s a reason most of these job interviews turn into algorithm hazing.

And my articulation of it was rooted, for me at least—at least in a small way—in a sense of defensiveness and wanting to validate the thing that I had done with my career that I defined myself by, I was under threat. And obviously, the thing that I do is the best thing because otherwise it’s almost a tacit admission that I made poor career choices at some point. And I don’t think that’s true, either. But for me, at least psychologically, it was very much centered in that. And honestly, I found that the right answer for me was, in fact, neither of those two things because I have met a couple of people in my life that I would consider to be full-stack engineers.

And there’s a colloquialism these days, that means oh, you do front-end and back-end. Yeah. The people I’m thinking of did front-end, they did back-end, they did mobile software, they did C-level programming, they wrote their own freaking device drivers at one point. Like, they have done basically everything. And they were the sort of person you could throw any technical issue whatsoever at and get out of their way because it was going to get solved. Those people are, as it turns out, the best. Like, who does a better job developer or operations, folks? Yes. Specifically, both of those things together.

Maish: Exactly.

Corey: And I think that is a hard thing to talk about. I think that it’s a hard—it’s certainly a hard thing to find. It turns out that there’s a reason that I only know two or three of those folks in the course of my entire career. They’re out there, but they’re really, really hard to track down.

Maish: I completely agree.

Corey: A challenge that I hear articulated in some cases—and while we’re saying things that are going to get us yelled out on social media, let’s go for the fences on this one—a concern that comes up when talking about enterprises moving to cloud is that they have a bunch of existing sysadmin types—while we’re on the topic—and well, those people need to learn to work within cloud. And the reality is, in many cases that first, that’s a whole new skill set that not everyone is going to be willing or able to pick up. For those who can they have just found that their market rate has effectively doubled. And that seems, on some level, to pose a significant challenge to companies undergoing this, and the larger the company, the more significant the challenge.

Because it’s my belief that you pay market rate for the talent you have whether you want to or not. And if companies don’t increase compensation, these people will leave for things that double their income. And if they raise compensation internally, good for them, but that does have a massive drag on their budget that may not have been accounted for in a lot of the TCO analyses. How do you find that the companies you talk to wind up squaring that circle?

Maish: I don’t think I have a correct answer for that. I do completely agree—

Corey: Oh, I’m not convinced there’s a correct answer at all. I’m just trying [laugh] to figure out how to even think about it.

Maish: I… have seen this as well in companies which I used to work for and companies that were customers that I have also worked with as part of my tenure in AWS. It’s the fact of, when companies are trying to move to the cloud and they start upskilling their people, there’s always the concern in the back of their mind of the fact, “Okay, I’m now training this person with new technology. I’m investing time, I’m investing money. And why would I do this if I know that, for example, as soon as I finish this, I’m going to have to just say, I have to pay them more because they can go somewhere else and get the same job with a better pay? So, why would we invest amount of time and resources into upscaling the people?”

And these are questions which I have received and conversations which I’ve had with customers many times over the last two, three years. And the answer, from my perspective always, is the fact is because, number one, you’re making the world a better place. Number two, you’re making your employees feel more appreciated, giving them better knowledge. And if you’re afraid of the fact of teaching somebody to become better is going to have negative effects on your organization then, unfortunately, you deserve to have that person leave and let them find a better job because you’re not taking good care of your people. And it’s sometimes hard for companies to hear that.

Sometimes we get, “You know what? You’re completely right.” Sometimes I don’t agree with you because I need to compete there, get to the bottom line, and make sure that I stay within my budget or my TCO. But the most important thing is to have the conversation, let people hear different ideas, see how it can benefit them, not only by giving people more options to maybe leave the company, but it can actually make their whole organization a lot better in the long run.

Corey: I think that you have to do right by people because reputations last a long time. Even at big companies it becomes a very slow thing to change and almost impossible to do in the short term. So, people tell stories when they feel wronged. That becomes a problem. I do want to pivot a little bit because you’re not merely an EntReloper; you are an EntReloper specifically focusing on container services.

Maish: Correct.

Corey: Increasingly, I am viewing containers as what amounts to effectively a packaging format. That is the framework through which I am increasingly seeing. How are you seeing customers use containers? Is that directionally correct? Is it completely moonbat stuff compared to what you’re seeing in the wild, or something else?

Maish: I don’t think it’s a packaging format; I think it’s more as an accelerator to enable the customers to develop in a more modern way with using twelve-factor apps with modern technology and not necessarily have their own huge, sticky, big monolith of whatever it might be, written in C# or whatever, or C++ whatever it may be, as they’ve been using up until now, but they now have the option and the technology and the background in order to split it up into smaller services and develop in the way that most of the modern world—or at least, the what we perceive as the modern world—is developing and creating applications today.

Corey: I feel like on some level, containers were a radical change to how companies envisioned software. They definitely provide a path of modernizing things that were very tied to hardware previously. It let some companies even just leapfrog the virtualization migration that they’d been considering doing. But, on some level, I also feel like it runs counter to the ideas of DevOps, where you have development and operations working in partnership, where now it’s like, welp, inside the container is a development thing and outside the container, ops problem now. It feels almost, on some level, like, it reinforces a wall. But in a lot of cultures and a lot of companies, that wall is there and there’s no getting rid of it anytime soon. So, I confess that I’m conflicted on that.

Maish: I think you might be right, and it depends, of course, on the company and the company culture, but what I think that companies need to do is understand that there will never be one hundred percent of people writing software that want to know one hundred percent of how the underlying infrastructure works. And the opposite direction as well: that there will never be people which maintain infrastructure and understand how computers and CPUs and memory buses and NUMA works on motherboards, that they don’t need to know how to write the most beautiful enticing and wonderful software for programs, for the world. There’s always going to have to be a compromise of who’s going to be doing this or who’s going to be doing that, and how comfortable they are with taking at least part of the responsibility of the other side into their own realm of what they should be doing. So, there’s going to be a compromise on both sides, but there is some kind of divide today of separating, okay, you just write the Helm chart for your Kubernetes Pod spec, or your ECS task, or whatever task definition, whatever you would like. And don’t worry about the things in the background because they’re just going to magically happen in the end. But they do have to understand exactly what is happening at the background in the end because if something goes wrong, and of course, something will go wrong, eventually, one day somewhere, somehow, they’re going to have to know how to take care of it.

Corey: I really want to thank you for taking the time to speak with me today about, well, I guess a wide ranging variety of topics, some of which will absolutely inspire people to take to their feet—or at least their Twitter accounts—and tell us, “You know what your problem is?” And I honestly live for that. If you don’t evoke that kind of reaction on some level, have you ever really had an opinion in the first place? So, I’m looking forward to that. If people want to learn more about you, your beliefs, call set beliefs misguided, et cetera, et cetera, where’s the best place to find you?

Maish: So, I’m on Twitter under @maishsk. I assume that will be in the [show notes 00:26:31]. I pontificate some time on technology, on cooking every now and again, on Friday before the end of the weekend, a little bit of politics, but you can find me @maishsk on Twitter. Or maishsk everywhere else social that’s possible.

Corey: Excellent. We will toss links to that, of course, in the [show notes 00:26:50]. Thank you so much for being so generous with your time. I appreciate it.

Maish: Thank you very much, Corey. It was fun.

Corey: Maish Saidel-Keesing, EntReloper of container services at AWS. I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated it, please leave a five-star review on your podcast platform of choice along with an angry comment that your 5000 enterprise developer colleagues can all pile on.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Announcer: This has been a HumblePod production. Stay humble.