Many Agile methodologies promote a daily standup meeting. The idea is that people are uncomfortable standing for a long period of time, so it will help keep the meeting short. The goal of the daily meeting (typically first thing in the morning) is to get a sense of what everybody is working on and if there are any obstacles. But, is the daily standup meeting effective?

I have worked for many companies that implemented some variant of Agile development and some of them implemented daily standup meetings as well. Based on my observations, the daily meeting is just a ceremony that doesn't really contribute much. Let's consider two different teams: small and large. The small team is generally fewer than five people. The large team is likely more than ten people. Teams that have between 5 and 10 will behave as either small or large depending on the team members and the culture. Small teams often work in the same room/open space. They already know what everybody does and obstacles can be addressed on the fly by turning around and talking to your teammate.

Large teams may be split physically and functionally and often work on multiple projects or separate aspects of the same project. The daily standup meeting either takes a long time when every member takes the time to explain what they're working on and what issues they face or is kept short by rushing through and having each team member mumble a few sentences. There is no time for questions and often people are not really interested in the daily struggles of others and just want to get on with their day. Any serious discussion requires a separate meeting with the relevant people.

Given those concerns, what are the benefits of the daily standup meeting? It can contribute to team spirit, serve as a launch pad to a more in-depth meeting and occasionally someone may have insight about how to overcome a particular obstacle. The team lead may gain some benefit by getting a concise summary from the whole team, and finally it ensures that everybody is in the office at the same time!

             

Agile development practices have become mainstream as large and small organizations recognize Agile development's superiority over other approaches for most projects. Organizations that follow the Agile method for development can produce real business value continuously. But, how do you deliver that value to your users?

Typical marketing departments will run large campaigns, sometimes synchronized with big industry events, sometimes with big launch events for important product releases. This is all in stark contrast to the development organization. If the developers added capability X to the product, why wait several months and deliver it to the users with capabilities Y and Z together. Yes, it makes for a more impressive presentation when you show X, Y and Z. But, you actually left value on the table by not delivering X when it was ready.

Agile marketing is all about being in tune with the development organization and also the market. Some of the values of Agile marketing are:

• Big plans are eschewed in favor of responding to changes
• Many small experiments over a few large bets
• Discovery over prediction
• Adaptive and iterative campaigns over big-bang campaigns

There are many benefits to this approach since the marketing organization is in tune with both the customers and the development organization. This allows communicating changes or better understanding of user needs to the developers in a continuous fashion (build the right thing). The users in turn are delighted because their wishes and requests are getting satisfied much more quickly.

The lack of big announcements about upcoming future products and capabilities takes away the risk of committing early to something that is not well understood or possible given current technology. As a result, users will not be disappointed when you fail to make good on your promises. This goes against the grain of the "establishment" but if you work in an organization that follows Agile development practices, consider pushing towards more Agile marketing as well. Take advantage of the great synergy that can exist between the various groups of your organization.

             

Agile methodologies have been used successfully in many big companies, but it is often a challenge. There are many reasons: lack of project sponsorship, prolonged user validation, existing policies, legacy systems with no tests - and most importantly culture and inertia. Given all these obstacles how do you scale Agile processes in a big organization? Very carefully. If you're interested in introducing Agile development practices into a large organization, you can try some of these techniques:

1. Show don't tell - Work on a project using Agile methods. Get it done on time and on budget using Agile methods.
2. Grow organically and incrementally - If you're a manager it's easy. Start with your team. Try to gain mindshare with your peer managers - for example, when collaborating on a project, suggest the use of Agile methods to coordinate deliverables and handoffs. If you're a developer, try to convince your team members and manager to give it a try.
3. Utilize the organizational structure - Treat each team or department as a small Agile entity. If you can, establish well-defined interfaces.
4. Be flexible - Be willing to compromise and acknowledge other people's concerns. Try to accommodate as much as possible even if it means you start with a hybrid Agile process. Changing people and their habits is hard. Changing the mindset of veteran people in big companies with established culture is extremely difficult.

Finally, if you are really passionate about Agile practices and everything you've tried has failed, you can always join a company that already follows agile practices, including many companies from the Fortune 2000.

             

Two Meanings?

1. The form of internal documentation appropriate for an organization following agile practices.
2. Generating external documentation as an artifact/user story.

The first meaning is typically a combination of code comments and auto-generated documentation. A very common assertion in Agile circles is that unit tests serve as live documentation. Python, for example has a module called doctest in which the documentation of a function may contain live code examples with outputs that can be executed as tests which verify the correctness.

Behavior Driven Development

BDD is putting a lot of emphasis on even specifying the requirements in an executable form via special DSLs (domain specific languages), so the requirements can serve as both tests and live human readable documentation. Auto-generated documentation for public APIs is very common. Public APIs are designed to be used by third party developers who are not familiar with the code (even if it's open source). The documentation must be accurate and in sync with the code.

The second meaning can be considered as just another artifact. But, there are some differences. Typically, when generating external documentation for a system it is centralized. You have to consider the structure and organization and then address the content as a collection of user stories. Unlike code artifacts, external documentation doesn't have automated tests. Documentation testing is an often neglected practice. Which is fairly typical because the documentation itself is often neglected. However, some types of external documentation are critical and must serve contractual or regulatory requirements. In these cases, you must verify that the documentation is correct.

             

Agile practices have proven themselves time and again for development and evolution of software systems. But, it's not clear if the same agile approach can benefit user-facing aspects such as public APIs, user interface design and user experience. If you change your API constantly, no sane developer will use it. If your user interface design or experience keeps shifting users will get confused and angry that they have to face a new learning curve whenever you decide to make a change. Sometimes, users will be upset even if the changes are demonstrably beneficial, just because of switching costs. Remember users didn't subscribe to your agile thinking and are just interested in using your API/product.

What's the answer then? Do you have to be prescient and come up with the ultimate API and user interface right at the beginning? Not at all. There are several solutions that will allow you to iterate here as well. But, you have to realize that iteration on these aspects should and will be slower and more disciplined.

Possible approaches include A/B testing, keeping the old API/interface available, deprecating previous APIs, backward compatibility, testing rapid changes on groups of users that sign up for beta. In general, the more successful you are, the less free you are to get rid of legacy. Probably the best example is Microsoft which still allows you to run DOS programs on the latest Windows versions and used a variety of approaches to iterate on the Windows desktop experience, including handling the frustration from users whenever a new version of Windows comes out. Windows 10 is a fine response to the harsh criticism Windows 8 endured.

             

Micro service architectures gain a lot of mindshare recently for good reasons. Let's assume you did the hard work and you have switched your system to use micro services. What are the benefits to an agile team? What would an agile micro service looks like?

The best thing about a micro service is that it is micro. It has a very well scoped functionality and it's easy to wrap your head around what it's doing. That means it is very easy to deliver significant features or improvements to a micro service within a single sprint. It also means that it is easy for a new engineer that has never worked on the micro service to pair with another engineer and quickly get up to speed.

The interactions with other services are through well-defined interfaces, so it is easy to use dependency injection to mock them and create automated unit tests. The narrow scope of the micro service helps to also to ensure that the test coverage is adequate. Refactoring anything inside the micro service is very simple because there is no need to coordinate with other services. It also aids in scaling a system incrementally.

At some point, every assumption you made when you designed the smaller system will come back to bite you when you scale — that data fits in memory, the data fits on the hard disk, the data can be stored reliably in one data center, the server can return all the data without timing out, etc. One approach is to try to think about all these eventualities and engineer them into the system from the beginning. This is typically a very bad decision. Your system might not scale to the level of engineering you built in for years, the time to market will be significantly higher if you try to build infinitely scalable system from the start, operations and maintenance will be more difficult. But, the worse thing is that when your system actually scales to the levels that require such engineering effort you'll discover that your well-crafted super-engineering from three years ago completely missed the mark and you have to start from scratch.

Micro services can help with that too. You can often isolate scalability, fault tolerance and high-availability into dedicated micro services that you build later and weave with your existing business logic micro services. The loosely coupled nature of micro services lends itself very well to agile evolution of the system.

             

Agile development, when executed well, is a thing of beauty. A sprint starts, stories/tasks are assigned. People start working in a test-driven manner and the system comes to life piece after piece. With each sprint new functionality is delivered. Refactoring is ongoing. The system architecture is always close to the conceptual ideal. Stakeholders are happy. Developers are happy. Everybody is happy.

And then, there is the real world--pressure, mid sprint changes, unclear requirements, tests without much coverage, technical debt mounting, production systems crashing, etc. With agile development that's not supposed to happen, but agile practices require dedication and discipline from all stakeholders--including top management, which is not always there. Even at development team level, some people are not fully on board or are simply not organized enough.

There are some inglorious tasks such as heavy documentation, verifying compliance with obscure standards and checking backward-compatibility with Netscape Navigator 4 on Windows 95, for example. If you reach a point where you notice Agile fatigue where basic practices are not followed, you can try gamification. Gamification is all about creating a work environment where you get rewarded for performing your duties by incorporating game design elements.

Whenever you squash a bug you get some experience points, the system keeps track of your not being late for meetings streak, the whole team gets a gold star when all of the sprint is completed successfully. This may sound silly if you never tried it, but movements such as the quantified self demonstrate that people enjoy it and will adapt their behavior to accomplish little game goals that just happen to correspond to real life goals. It can add fun, as well as help keep track on actual important metrics.

             

Test Driven Development (TDD) is arguably the most impactful Agile practice. Nobody even talks about it anymore, but automated tests were revolutionary when they came on the scene. Fifteen years ago, the common practice was to have a big monolithic application. The developers would produce a build (usually an executable) once in a blue moon, throw it over the fence to the QA department which would bang on it for a while and then open tens if not hundreds of bugs. Then, the developers would engage is a bug squashing period.

Luckily, that's no longer the case and everyone recognizes the importance of automated tests and rapid iterations. TDD means that your whole development process is driven by tests. When you think about problems you phrase the discussion in terms of what tests you'll need. When you model things you consider how testable they are. You might change choose between alternatives based on the impact on your test suite.

The ultimate in TDD is the test first movement. This is truly putting tests on a pedestal and treating tests as the most crucial building blocks of your system and its architecture. There are multiple advantages to TDD. The best one is that you get to say TDD all the time. Try it. It's fun: TDD, TDD, TDD. Almost as important is the fact that you will have automated tests for all your code and those tests will even be kept up to date as you evolve and refactor your system. Don't underestimate TDD. Every development team starts with the best intentions regarding all kinds of best practices, but as the pressure mounts many of them go down the drain: test coverage, modularity, coding conventions, refactoring, clean dependencies, etc.

Having tests as your primary driver ensures they will not be neglected, which will give you better a chance at evolving your systems.

             

Agile practices work best with small cohesive and co-located teams. How do you scale it for a larger organizational with multiple teams, possibly in different locations and even time zones? One approach would be to treat all these teams as one big agile team and try to work around the issues of remoteness and time zones. This approach is doomed for multiple reasons.

A better approach would be to break the development into vertical silos in which each team is completely responsible for a particular set of applications or services. This approach scales well as long as you can neatly break development into independent pieces that can be fully owned by one team. One downside of this approach is that knowledge sharing and opportunities for reuse are much more difficult.

A third approach would be to treat teams as users of each other. This allows collaboration and combining multiple teams to tackle big projects without losing the benefits of the original Agile team. This requires careful management to ensure teams' schedules are properly coordinated and dependencies don't halt development. This is nothing new, but in an agile context the planning game is typically done at the team level. When multiple teams work together (even loosely) there needs to be another layer of management that takes care of the cross-team planning.

             

Open source development has never been more prevalent and successful than it is today. The biggest players regularly publish the latest and greatest technology with permissive licenses. Some companies even do their whole development in public — inviting anyone to fork, download and send pull requests.

Google has always been a strong advocate of open source and even funded a lot of non-Google open source projects through their Summer of Code program. Facebook publishes not just software, but even the specs to their servers.

Microsoft recently joined the party and open-sourced key parts of their technology stack and the company even develops core pieces openly on Gitgub.

The big question is whether or not the agile development style mixes well with open source. On the face of it, they are polar opposites. Agile evolved from tightly knit co-located teams. Open source is all about strangers who have never met collaborating across the globe.

All the same, there are many similarities and shared principles between the two methodologies. Both paradigms put a great deal of emphasis on testing and automation. Both favor small and quick iterations. Obviously, co-located Agile teams can publish the results of their work as open source. The more interesting case is when an agile development team, that is not co-located, can still successfully take advantage of open source methodologies such as rigorous source control policies, pull requests and continuous integration tools.