CDT Doug

And Now For Something Completely Different

2020-09-29T02:00:00+00:00

Recently, I was inspired by Ben Eater’s YouTube serices where he built a CPU using 7400 logic chips on breadboards. I had thoughts of doing the same, but I wanted it to albe to demo well to kids at Maker Faires, play games on a VGA display using serial game controllers, for example. Well as the idea grew, the design was all of a sudden going to take 20 breadboards and probably more. That’s like $200 worth of boards.

Meanwhile, I had seen articles on how hobbyists were building soft CPUs that did things like emulate the venerable Commodore Amiga. It’s really only recently that FPGA boards for the hobbyist market are starting to be powerful enough and cheap enough, and yes, are well under the $200 I’d have to spend on breadboards alone.

As I worked through the idea, I was energized. What a project it would be to build a computer from the logic gates on up to a working gaming rig. There’s lots of that going on in the 8-bit world from the 80’s. I’m more interested in the era of the early 90’s. I had a Commodore Amiga and I truely thought it was far better than the PC’s of the time. The Motorola 68000 has a much cleaner architecture than the 8086. The Amiga had a color GUI for crying out loud.

So I’m really curious what a m68k machine, of the early 90’s that rivaled the i486 that ran DOOM and changed the gaming world, would have ran DOOM and changed the gaming world. So that’s the plan. To build it and share it’s design as an education platform so others can see and learn how things we’re built back then, pre Windows 95 when the industry lost it’s pioneer spirit. It was so exciting back when the Apple II, TRS-80, Commodore 64 and so many other innovative machines fought tooth and nail for the minds of users and developers. What if that kept going into the 90’s?

I recently bought a ULX3S FPGA board from Crowd Supply produced by a group in Croatia. It has every I/O I could want to build my dream 90’s machine, plus some goodies that allow it to use modern peripherals, especially USB for keyboard and game controllers, HDMI for display output, and a slot for an SD card for storage. And not only that, they strapped an ESP32 module onto it to give it Wifi and Bluetooth. And all for what I was going to spend on my little breadboard computer. I can’t wait to get started.

EclipseCon 2018 and the New CDT

2018-10-29T22:00:00+00:00

EclipseCon for me is many things. It’s a chance to meet face to face with my fellow CDT contributors. It’s an opportunity to run things by one another that may feel awkward over the mailing list or conference calls. It’s a chance to get a good feel for what’s happening in the rest of the Eclipse IDE and the rest of the Eclipse ecosystem. And it’s a chance to hang out with my brothers and sisters in the community and have a few laughs over a few beers going too late into the night but ready to get to work the next morning. It’s the best.

This year was special for another reason. The Eclipse IDE is changing. The world of IDEs is changing. A new generation is upon us. And, no, it’s not any particular IDE. Nor is it my fictional Eclipse Two IDE :). And believe it or not, it does involve and give a new lease on life to the old workhorse most of us simply call Eclipse. It’s a new architecture for all IDEs and the Eclipse community is taking a leadership role in adopting that architecture. Talks on the topic were everywhere at EclipseCon.

(originally https://code.visualstudio.com/assets/docs/extensions/example-debuggers/extensibility-architecture.png)

Of course I’m talking about the Language Server Protocol and the Debug Adapter Protocol. They were introduced by Microsoft for Visual Studio Code but are also open for adoption by any IDE. It allows users to chose the front end that gives them the best user experience while giving access to the language and debug features they expect from all IDEs. It allows IDE builders to work together on these features and it allows platform vendors to not only help with that and but also give their users and customers choice.

For the CDT, we’ve been monitoring the clang/LLVM based language servers closely. clangd has industry momentum but is missing some key features. cquery has a ton of features including extensions to the LSP but has a relatively small community. CDT is working on support for both by leveraging the common Eclipse LSP4E plugins and the Generic editor. We have a long way to go before these services reach parity with the current CDT, but working with this larger community, I’m confident we’ll get there. And it will solve our problem of keeping up with the ever evolving C++ language standard thanks to the great work that goes into clang.

On the debug side, we’ve seen a few open source gdb debug adaptors, but none of them are really suitable to the general CDT audience. So to help with that, my collegues at QNX will work with others in the community to build a gdb debug adapter as a part of the CDT. This will be our first foray into the JavaScript/TypeScript world as an Eclipse project. We’ve built up a tonne of expertise on gdb/IDE integrations over the years and I think we can do a pretty good job of it. And of course we’ll make it available for any IDE to use, which in turn means we’re open to any help other IDE providers can offer.

The biggest benefit of this new component architecture is to allow users choice. For CDT, we’re going to turn that on it’s ear a bit. For us, it’s also about sharing our expertise with other IDEs. Our first step down that road will be to produce a set of Visual Studio Code extensions first for our debug adapter to ensure a seamless experience on par with CDT. Depending on what happens on the language server side, we may also produce one for LSP to help integrate clangd which may need to be forked to properly handle gcc-based environments or add features the clangd community aren’t interested in.

Our committment has always been to provide the best open tooling for C/C++ developers. For many, many years, that was Eclipse. This new architecture opens the door for alternatives and as the C/C++ community spreads their wings into this new world, we, the CDT contributors, will be there for them.

C++ and TypeScript - The Next Generation is Now

2018-09-10T04:00:00+00:00

We’re coming up on the 10’th anniversary of one of my favorite EclipseCon memories, Eclipse Summit Europe 2008. Something about staying up until 5 a.m. local time with a bunch of messed up Canadians and Europeans who just didn’t want to go to bed that make an event legendary. What made that special was probably the audience with a legend himself, Dave Thomas, who arguably started this all and who stayed right with us.

But it was his keynote, “Next Generation Embedded Software - The Imperative is Agility!” that I drew a lot of inspiration from. I can barely remember what it was about but he seemed to be bashing Java a lot so I had somewhat tuned it out. Until the last slide or so where he presented the alternative he controversially suggested would be the next generation: C++ and JavaScript. Wait, what?

I was probably the only person in the room who took that seriously. It was the year that Google had introduced the V8 JavaScript virtual machine. I spent a little time after the conference figuring out how you’d use it with a C++ application and it definitely seemed plausible. What was missing was a user interface and that seemed like an enormous amount of work so I left it aside.

Fast forward a few years, we began to see people try and solve that problem by slapping node.js and browser together to create a desktop application framework. First, we had NW.js who started with webkit but at some point switched to Chromium. Now we also have Electron who have done the same but with more separation between node and Chromium to make it easier to keep up with releases of both, in theory.

Playing with Electron a couple of years ago I started to get the feeling that this vision was indeed coming soon, especially the first time I got Electron to load a C++ Node Addon. And now, I just used a C++ addon to solve a performance problem I was having in a VS Code extension (Electron incarnated as an IDE). That sealed it for me.

To show what I mean, I have taken the environment I’m using and created a really simple VS Code extension that has everything one would need to get started. It’s dumb, but it implements an asynchronous native function to add two numbers and shows the results in a VS Code webview panel. It includes a bit of messaging framework to allow for type-safe(r) messaging between the webview and the extension. As usual, it’s available on my github to check out.

That was a bit of a long winded introduction, but I’ll dive into the technical details for the rest of this article. Needless to say, I’m pretty excited about it.

Believe Me, It Builds

Figuring out the cleanest way to build this thing and bundle everything together was the most challenging part, and probably what I’m most happy with. There are three different platforms at work and I am able to build them all with a single ‘yarn compile’. And I’m able to work incrementally with minimal fuss, pretty much none, a watch, for for the extension and view and a simple incremental build for native on demand. And thanks to VS Code’s TypeScript support and the clangd C++ language server, I find errors even before I build, as it should be.

CMake

Let’s start with the native side. Everyone who knows me from my work on the Eclipse CDT know I’m all about the CMake build tool these days. After years of working with Makefiles that model the system file by file, CMake lets you work at a higher level where you specify executables and libraries and dependencies between them. It takes care of the file level dependencies for you with a lot of magic, but good magic.

To build against the node.js APIs, you also need to download the header files, and for Windows, a library to build against. That’s tricky to do, so I borrowed a few ideas from the interweb and created a NodeModule.cmake file under the CMake folder. It takes care of downloading the and extracting the necessary tar ball and sets up the dependencies to it. That makes the CMakeLists.txt file as simple as setting the Electron version for the tar ball, the output directory, and then defining your module.

set(ELECTRON_VERSION 2.0.5)
set(NODE_MODULE_OUTPUT_DIR ${CMAKE_SOURCE_DIR}/out)

include(NodeModule)
add_node_module(n2wNative native.cpp)

That places your module, n2wNative.node in this case, into a platform specific bin directory in the out directory of the project ready to import into the extension.

Webpack

Webpack seems to be the leading build framework for web applications. I also noticed with the latest VS Code release they are using it now with it’s built-in extensions. Previously, I had a two step extension build, running webpack for client side and the TypeScript compiler for the extension. They both have file watchers but found the TypeScript one a bit flaky. So bringing them together into the same builder would make things a lot cleaner.

Webpack allows you to have multiple configs. I created one for the client side and one for the extension. The only real difference is the options for the TypeScript compiler to deal with the different modules systems between the browser and the extension which is running on node.js. I also chunked out the node_modules content for the clients since you are likely to have multiple of those.

The toughest issue I had was the ‘require’ loading the native module. It seems webpack tries to guess at location and changes the require to somewhere it thinks the module will be. But the whole idea of this module is that it’s platform specific and it needs to calculate which platform it’s on at run time. I finally ran into non_webpack_require which is a hacky way to tell webpack to leave things alone. The TypeScript doesn’t know about that magic so it needs to be declared.

declare function __non_webpack_require__(path: string): any;

export function add(x: number, y: number): Promise<number> {
    const native = __non_webpack_require__(`./${process.platform}/n2wNative`);
    return native.add(x, y);
}

The next issue I had to deal with was source maps. When I first got things running, VS Code with the Chromium debugger couldn’t figure out my breakpoints. I managed to stumble across a magic command in the Debug Output view, .script, which showed a detail view of the source maps. Webpack uses a magic URL webpack:// in the source maps it seems to be causing the confusion. Luckily, I found out you can control that.

output: {
    path: path.resolve(__dirname, 'out'),
    filename: '[name].js',
    libraryTarget: 'commonjs',
    devtoolModuleFilenameTemplate: '[absolute-resource-path]'
},

devtoolModuleFilenameTemplate lets you manage the path used for your source file. I guess webpack is designed more for web apps where using the absolute path is weird.

Time for a N-API

Node offers two API to hook up your C++ Addon, a scary one and a nice one. The scary one actually uses a lot of the V8 APIs and is heavy C++ and tends to tie you to a specific node release for some reason. Their newer simpler API introduce in version 8.0 is called N-API and thankfully VS Code finally switched to Node 8 in version 1.26. It’s a C API and is very reminiscent of Java’s JNI so I found it very easy to work with. It’s marked experimental in Node 8 but I don’t notice any major changes in version 10.

I didn’t originally intend to go so native. I was working on a tool that scanned a gigabyte size binary memory mapped file extracting data. I started doing that in TypeScript using node Buffers and found the performance surprisingly good, under 3 seconds to do a complete scan. But then I added an else clause to the main if statement and the time hit around 15 seconds. Really? Did I disturb something in JIT that killed performance? How do you even predict how that’s going to work.

Since I had written my own little memory map function using, I decided to rewrite the algorithm in C++ and ended up with around 1 seconds for the scan. Wow. Being a CDT guy, I know C++ and am pretty comfortable with it. And Modern C++ makes dealing with pointers pretty safe and collections fairly easy. For me, the performance gains were well worth it.

As I dug more into the features of N-API, I also discovered a hidden treasure. You can actually run your time consuming native algorithm on a worker thread and then use a JavaScript Promise to return the result when it finishes.

Basically, you create a struct to hold the arguments and result of the algorithm as well as the ‘work’ object the API gives you and a ‘deferred’ object that you use to resolve or reject the promise. That struct gets passed to a ‘work’ function that runs in a worker thread. When the work finishes a ‘completed’ function runs on the JavaScript main thread where you convert the result to a JavaScript value and send it out the promise through the deferred object. Easy, peasy, and pretty powerful.

I won’t reproduce the whole file here, but check it out at src/native/native.cpp. It’s a bit wordy written in C but I imagine one could wrap it in some C++ classes and generics to make it easier.

Playing it Safe

When you spend most of your life working in typed languages, first C++ then Java, the prospect of writing a serious app in a dynamically typed scripting language like JavaScript is a bit unsettling. Luckily I’m jumping into this as TypeScript is maturing. It has really helped me, especially as I learn both the VS Code API and the various npm packages I’m using thanks to the great IDE features you get with typed languages.

But there are two areas where you end up interacting with the raw JavaScript environment that you have to manage. The first is the native module. Most modules that you get off npm come with TypeScript definition files to make the integration easy. You have to provide something like that for the native module.

I already showed the TypeScript code to do this above. The require call returns a JavaScript object that you create in the module init native function. There are magic ways to give that object a type but I found it easier to start to just wrap it with a TypeScript function that manages loading the module and making the function call, at least for now.

The other area we need to manage is the communication channel between the extension and the code running in the webview. The webview extension API let’s you bind a OnReceivedMessage callback that gives you the message as a JavaScript object. The webview client injects ‘message’ events into the window object where you can pick them up with a listener.

To make the messages type safe, I leveraged TypeScript’s Discriminated Unions to declare message types in the shared messages.ts file. On the client side, I created a ServerPort class to manage the client side where I declared override methods to make posting messages type safe. The server side just has a switch statement where TypeScript figures out the type based on the case.

    async receiveMessage(request: Request) {
        switch (request.command) {
            case 'addRequest':
                this.postMessage({
                    command: 'addResponse',
                    token: request.token,
                    result: await add(request.x, request.y)
                });
                break;
        }
    }

That’s the great thing about TypeScript. You still have the ability to do JavaScript when you need. But you have to come up with a strategy to minimize that surface area as much as possible.

Challenges Ahead

Now that I have this running in Visual Studio Code, I don’t see any reason why you couldn’t use this architecture in other environments. It is just node.js and a browser in the end.

For example, I could simply get my tools running directly in Electron as a stand alone offering for people who don’t want to use VS Code as an editor.

Web IDEs could also benefit. The Theia IDE promises to be compatible with VS Code extensions. It’ll be interesting to try since this is probably the most complex extension I can think of. But other IDEs that have node.js as a server could also work this way.

But one thing is clear. I’m excited! Whether other tools developers will be as excited isn’t as clear. Either way, I’ve found my next generation tools platform. It uses C++ and JavaScript/TypeScript. And that next generation is now.

Aliens, Go Home! VS Code-style!

2018-08-31T18:01:00+00:00

Looks ridiculous, doesn’t it. There is a method to my madness though. Stick with me…

I make it no secret how enthusiastic I am about Visual Studio Code as an IDE platform. I have often commented on my desire to start building tools using web front end technologies (and not necessarily web back ends). I even prototyped an “Eclipse Two” that was built on Electron directly. In the end, the Microsoft Zurich team, who also happened to be former leaders at Eclipse, created something similar with a huge community and ecosystem of extensions and I jumped on the bandwagon.

Being a good code editor and debugger is one thing, in the end there’s more to life as a developer than writing code. Often the systems we work with are more complicated than can be simply represented in text. We need graphics that can abstract away some of the gory details and make system behavior and relationships easier to understand. And that includes both ends, from modeling when creating the system, to tracing when trying to see what it’s doing once we built it.

There’s a number of ways you can do graphics in a web front end. The first one I considered was SVG. I like it because it’s backed by a data model with API that will let you change properties programmatically. And it simply gives an object oriented approach to graphics. It’s not all good though since those objects come with a price in memory and setup time. But if you keep the number of elements you create under a thousand or so, it’s pretty quick.

The other advantage of being in the DOM is that React can handle SVG with it’s virtual DOM. This speeds up rendering from changes to the underlying data model and automatically calculates that for you. It’s one of the coolest things in React and why we use it for all our web UI work.

So I set off to google around for examples where people had done that and I quickly ran across an article series Developing Games with React, Redux, and SVG. It’s really cool and proves out my main thesis about handling changing data models. Games, of course, are all about changing data models!

To complete the exercise, hooked up a simple VS Code extension that opened up a webview panel and ran the game in it. I rewrote most of it using TypeScript, since I refuse to give into the typeless world of JavaScript. I also switched over to MobX which some of my colleagues were using and it’s much easier to deal with than Redux. The whole thing was a joy to work with.

If you’re interested, I have pushed up my results to my GitHub repositories. It does look ridiculous running in VS Code. But it shows off the power of the platform. Running in a webview means you’re running in a separate process from the rest of vscode so what ever you do there doesn’t impact the rest of the IDE. From there, you can communicate back with your extension to allow it to interact with your environment and grab data from systems and files and feed back data for rendering. I can’t wait to see where the community (and myself for that matter) take this architecture.

Using CMake for the ESP32 with the Eclipse C/C++ IDE

2018-07-02T18:00:00+00:00

The Photon release of the Eclipse C/C++ IDE now includes support for a number of different build systems. CMake is one of the most popular build systems with open source projects so it was an obvious candidate. The Eclipse CDT project has been working on CMake support for a few years and now it’s ready for wider adoption.

One of the open source projects using CMake is the Espressif IoT Development Framework, ESP-IDF, an SDK for their inexpensive and highly featured system on a chip, the ESP32. It is widely popular with hobbyists with boards available from a number of vendors. Though CMake support is currently in a feature branch, it is expected their CMake build files will make it to the master branch in the upcoming months.

This environment is a great example of how the new features in the Eclipse C/C++ IDE automate much of the build and launch process to speed up development on microcontroller projects. We will walk though each of the steps to build an application and get it running on a board and will highlight the new features in Eclipse Photon as we go.

Environment

Espressif provides thorough documentation on how to set up your development environment. Here’s a quick highlights of what I did for mine. I’m using Windows 10 with CMake, Ninja, and Python 2.7 provided by scoop. No MSYS or Cygwin. I use busybox for my shell but Eclipse doesn’t need it.

Follow the Setup Windows Toolchain from Scratch instructions to get the xtensa toolchain and mconf Kconfig utility. Add them to your PATH environment variable.
Checkout the features/cmake branch of the ESP-IDF. Make sure you do a recursive checkout to pick up it’s numerous git submodules. Set the IDF_PATH environment variable to the root location of the repo.
Install the pyserial Python library for esptool.

Now we’re ready to fire up Eclipse.

CMake Project

In the New C/C++ Project wizard select “Empty or Existing CMake Project” from the list of templates and create a new project. This will create the project but without a CMakeLists.txt or any source files.

The Toolchain

We now need to tell CDT which toolchain to use when building the project. This will pass the required arguments to CMake when generating the Ninja files.

In the Preferences, select C/C++ and then CMake to see the list of toolchain files CDT will use with CMake. Click Add. Each CMake toolchain file is associated with a toolchain. Since we have the xtensa toolchain in our PATH, CDT will auto discover it and it will appear in the dropdown. Also enter the location of the esp-idf toolchain file which is in the esp-idf repo under tools/cmake/toolchain-esp32.cmake.

The Launch Target

Next we need to tell CDT to use the toolchain for our project. This is accomplished through the Launch Bar, the new widget set you see on the far left of the toolbar.

Click on the third dropdown and select New Launch Target. We need to tell CDT about the target where we’d like to launch our application. We’ll be using the flash loader so select Serial Flash Target. Enter a name for the target, “esp32” as the operating system, “xtensa” as the CPU architecture, and select the serial port your ESP32 device is connected to on your machine.

The OS and architecture need to match the settings for the toolchain. You can see those settings in the Preferences by selecting C/C++ and Core Build Toolchains. For GCC toolchains, CDT autodetects those settings by asking GCC for it’s target triple.

Build

For this first project, we’ll use the Hello World example from the ESP-IDF. It is found in the examples/get-started/hello_world. Copy the contents of this directory into your project. Since we’re using CMake you can delete the Makefile and the main/component.mk files to prove you’re using CMake.

Notice that the Launch Bar shows Run Hello World on ESP32. When you click the Build button, it will run a build of that project for the ESP32 Launch Target in Run mode, the only one available for the ESP32 for now, and will make sure the right toolchain and CMake toolchain file are used in the build.

Terminal

To see what our program does, we need to see the output from the serial port. I have an Adafruit Huzzah32 which uses USB to both power the board while in development and to talk over the serial port. We’ll use the Eclipse Terminal to connect to that serial port. Click the Terminal toolbar button and select Serial Terminal as the terminal type and complete the rest of the settings.

Launch

To get the app running on the board, we need to download it over the USB serial port and burn it into the flash memory on the device. The ESP-IDF has a tool for that called esptool. The IDF also has a nice wrapper script that manages flashing as well as a number of other handy operations.

To hook it up, we need to set up the launch configuration. This is where the Launch Bar shines. Click on the gear next to your project in the middle selector. It will open a launch configuration edit dialog. Since our target type is Serial Launch Target, it has a simple launch that executes an external tool. We’ll set it up to run python with the idf.py script and set the build directory and serial port and ask it to flash.

If you had a program running on your chip you’ll notice in the Terminal that it stopped printing there. The CDT has noticed that your launch target and the terminal are using the same serial port and pauses the terminal while the python script runs. This is especially important on Windows since only one program may control the serial port at a time. Once it starts up again, you see the output of your program. This makes it very easy to iterate changes.

Conclusion

The Eclipse Photon release of the C/C++ IDE is the culmination of a number of years of work on the Launch Bar and CDT’s new Core Build system that gives us CMake and makes it easy to integrate other build and launch tools. Bringing these tools together and having them work together to accellerate development is what Integrated Development Environments are all about.

Get the Scoop, a Homebrew for Windows

2018-06-23T00:00:00+00:00

When I came back to QNX over six years ago (wow, it’s been that long?), they offered a choice of one of the the three main environments. I was excited to see what all the hype was about and picked Mac. I really enjoyed it. It provided a great blend of user experience with the power of Unix and the shell underneath. The trackpad on the MBP is amazing. It’s under rated how much a productivity enhancer that is.

But eventually that machine got old and beat up and it was time for a new one. I had my fun with the Mac, but it’s not what’s used by many of the users of the tools I build. In the embedded space you see a pretty solid mix of Windows and Linux. And I was interested to see how much better Windows 10 was and whether it could overcome it’s really crappy command line environment. So that’s where I went.

I knew Eclipse works well there. It was invented there and still looks like a big MFC app. But selecting a shell environment revealed a lot of choice.

MSYS. This was the environment I used the last time I was on Windows. It came as the shell for the MinGW toolchain. But it seems to have died.
CYGWIN. This environment bugs me a lot as it maps Windows paths to Unix paths and confuses the hell out of native tools, like Eclipse CDT.
MSYS2. This is a pretty rich environment that seems to be an evolution of the old MSYS. It comes with a package manager but the choice of Arch Linux’s pacman is a tough one. It makes it your responsibility to figure out what 32/64-bit host and target combinations of the toolchains and libraries you want to install. But it does have everything, even Qt.
Scoop. Scoop is a more general package manager and is pretty easy to use. It’s very active and has a good community keeping the tools up to date. And while it has all my favorite tools for building C/C++ apps, it doesn’t have any libraries. But that’s fine, you should be building those yourself anyway to make sure you’re using the same toolchain settings.
Chocolatey. This is another package manager but at a higher level than Scoop. It’s less focused on being a shell and assumes you’re a Powershell user, which I’m not and barely understand.
Windows Subsystem for Linux. Also know as Bash for Windows, it provide a Linux emulation layer and gives you access to real Linux distributions. You can use it to access you’re files outside of it’s Linux emulated file system, but it’s pretty weird and not everything works.

For now, I’ve gone with Scoop. I really like it because I also use it to install other tools I use like docker and kubectl for my test cluster, and maven and python and svn, pretty much any command line tool I need. And it manages the PATH for you so all the tools are available in native apps without any magic, including the Windows command line if you’re stuck for anything else. But most of the time, I use busybox which gives me all the Unix tools I need including a not quite bash compatible shell, but that’s fine. I do wish it had C/C++ libraries like SDL or Qt, but it is open source and extensible so I could just do this myself.

The good news is that I’m finding myself very productive on Windows. I have a great shell environment with Scoop, good editors with emacs and Visual Studio Code, and Eclipse which still works best on Windows, and all the Windows apps I need. I don’t miss my Mac at all.

Going Serverless

2018-05-10T00:00:00+00:00

Going Serverless

Well, after a couple of years running my cdtdoug.ca server on AWS and the couple of years before that on DigitalOcean, I finally got tired of managing my own server and have moved my blog over to Github Pages. I’m using the default Minima template to get me going, but reading through the Jekyll docs, it looks like I’ll have some fun tweaking my own setup the way I want it.

For the first 8 years of my blogging life, I had hosted it on Google’s Blogger. But being the foolhearty geek craving to learn new things, I got my own little server in the cloud and moved my blog there, first on Wordpress and then recently moved to the much lighter weight Ghost. I learned a lot and it was pretty fun but I think I got what I wanted out of it and it’s time to move on to the next thing.

The other reason for getting a server was to do some home IoT projects. I had visions of setting up my own MQTT broker there and a simple node.js web site to provide the UI. But the world has changed. I am now looking at serverless offerings that should allow me to do all this without having to manage a server and at a fraction of the cost of even the t2.nano. One device, a few thousand messages a month, a simple REST API and a tiny static web site. It’s almost free.

So that’s my next adventure. First, I’m hoping I’ve set up the Planet Eclipse feed to pick up the generated RSS XML file properly. Then I’m off to play with some new toys.

Building a Custom Viewer for VS Code

2018-02-24T01:12:47+00:00

I attended the online CheConf this week and was impressed by a few of the sessions. Now don’t get me wrong, I still have reservations over the practicality of having your IDE in the cloud, at least in the embedded space where engineers tend to be more conservative. Certainly running tools inside docker containers makes sense and I believe all IDEs, cloud and desktop, need to be able to interface with them and Jeff from Red Hat is making good progress with that in the CDT. So we’re not so different.

What did impress me from the sessions, though, is the progress we’re starting to see with advanced visualizations using Web front ends. I was especially excited to see Mélanie from Obeo show Sirius there. And Silexica is doing some interesting visualizations of call graphs of C++ code with what I believe is D3.js. The charts you see come out of D3 are amazing, especially with animation. Being able to visualize complex systems this way will be a huge help to developers, epecially embedded engineers.

As I mentioned at the beginning of last year, I’m very excited about Electron as a desktop IDE platform. After playing around a bit on my own, I decided to give up and start working with one that’s already well ahead, Visual Studio Code. Despite their claims to be focused on providing a great text editor, the community is working hard to make it a full fledged IDE. With the extensibility available today, you can do the same advanced visualizations we saw at CheConf as well as many other online tools.

I have created a demo of this capability as shown below. Click on the picture to see the github repo and the description in the README. It’s dead simple but it shows how you can create a VS Code extension that renders in a webview whatever you would like with data fed from an embedded http server. It’s a little weird, but once you get the plumbing done, you’re on your way, and hopefully this demo can help you get started.

There are many reasons why this move is important. There is so much momentum behind web technologies with a seemingly endless line up of libraries on npm, to tutorials on the web, to people who know how to do this stuff. It only makes sense that we start to ride this wave as we did when the Eclipse IDE first came along. And, you know, fire this stuff up in a SWT Browser widget and you get that same experience and we can breath some new life into our old friend. It’s an exciting future and it’s a lot of fun trying to get there.

What makes a good C++ IDE?

2018-01-28T18:31:34+00:00

When I started working on C code many years ago, I was a big fan of emacs. It didn’t have the weird modes that vi did. You just typed and code showed up. emacs has a rich extension system, if you don’t mind writing LISP code, and soon we had fancy features such as running builds and navigating through compile errors. There’s even a gdb integration that gives you what you need to debug your program. And then along came ctags and we started getting source navigation.

But what was good for C started to fade with C++. We also started bringing in code coverage and profiling tools and even modeling and code generation tools. Simple editors like emacs couldn’t keep up. We needed much more powerful platforms that provided a richer environment for tools as well as the promise of standard interfaces that allowed us to integrate tools to provide functionality greater than the whole. The IDE was born.

But somewhere along the way it seems we failed. We didn’t provide enough value that warranted the opinionated structures that IDEs enforce that enable the standard APIs to work. As I’ve stated many times, we have paid dearly with Eclipse trying to force C/C++ projects into the Eclipse resource system. What is a project in C/C++ world? Are projects the root of a recursive make tree, or the individual nodes that represent executable things and libraries? Neither works well, especially in complex systems embedded systems developers are building.

So after years of trying to convert die hards and working hard to support those who want to use our IDEs, we need to take a step back try again. Get back to emacs, or more modern editors like Visual Studio Code, and see how we can bring our users forward again. This time we need to do it in a way that doesn’t enforce a paradigm on them yet provides a path back towards the valuable features we can provide through integration.

Anyway, that was a long preamble towards the real topic of this post, remembering what made CDT successful and providing at least that first step forward. What is the minimal feature set that meets expectations of today’s developers. As I was using Visual Studio Code to hack Visual Studio Code, you get a real sense of what that is. You also get a good sense at what’s missing from these editors.

First of all is source navigation, being able to control click to find definitions of symbols and to be able to search for references to them. And, of course, along with that comes content assist. I spent most of my early years on CDT figuring out how to do this properly and quickly. For those that remember, the quickly took a long time and in the end exchanging perfect answers for speed was our path to success. To make this work we hand built C and C++ parsers that referenced a symbol database called the index. You also need to know the toolchain and build system to know how the code is built to do a reasonable job at it. Integration with the build system is a must.

It’s hard to imagine how editors like VS Code can be a good CDT replacement without replicating the magic. I’m not convinced the current front runner clangd and it’s use of the clang parser which has perfection as it’s number one requirement, will get us there without a lot of hacking. And editors usually don’t deal with builds other than to invoke them and give you error navigation to feed it the info it needs.

And I’m not even touching on launching where you really need to match build configs with launch configs to make sure launches do what you want. So while editors are all the rage right now because they are lightweight and flexible, users still expect them to more than the editors of old. It will be an interesting challenge for those who go that road to walk that balancing line. If you get it right though, I can imagine a lot of happy users.

Following the Community’s Lead

2018-01-03T21:35:22+00:00

I posted this tweet a couple of weeks ago and thought I’d explain it a bit more.

2018 is going to be a defining year. Either it'll be the end and a new beginning, or it'll be a revival. Not sure I control either path other than my desire to not walk it alone.
— Doug Schaefer (@dougschaefer) December 21, 2017

I wrote it shortly after posting the final bits for the CDT 9.4 release and was starting to think about my plans for 2018. And certainly the current state of Eclipse tools projects including the CDT has been weighing on my mind.

When QNX started the CDT project, we embraced open source as a means of sharing in a common effort to build a great C/C++ IDE. We didn’t have a huge team and we knew that many platform vendors were in a similar situation. By working together, we all benefit. And for many years it worked and we now have the embedded industry’s most popular IDE that we all should be proud of.

But times they are a changing. As I blogged last year around this time, web technologies are becoming the platform of choice, not just for web, but with the help of Electron, for the desktop as well. We can not discount the popularity of Visual Studio Code and the ecosystem that has quickly developed around it. And with shared technologies like language servers, the door is being opened to choice like we’ve never had before.

So as I was inferring from my tweet, it’s time to take stock of where we are and where we are going. I was encouraged with the discussions I had at EclipseCon around the future of CDT. I’ve done what I can to help keep things rolling but at the end of the day the community leads the way and I need to be honest with myself and my customers and go where they go.