Tag Archives: Linden Lab

Lab updates on viewer changes and CDN

secondlifeThe Lab has issued a blog post outlining some of the current improvements being made to Second Life.

Regular readers of my weekly SL project updates will already be familiar with the work referenced in the blog post, which focus on the changes being made to the viewer’s log-in screen, the removal of the viewer’s reliance on the GPU table when initially setting graphics preferences, the ongoing deployment of support for using a Content Delivery Network (CDN) for texture and mesh fetching, and an announcement of the upcoming HTTP pipelining viewer, which should offer some significant improvements in people’s SL experience, as well as including further adjustments to leverage the CDN.

Commenting on the new benchmark viewer, which will eliminate the need for the GPU table, the Lab’s blog post states:

This is a new way of figuring out the best default graphics settings. Maybe this has happened to you: you got an awesome new graphics card, fired up SL… only to discover your graphics settings are set to Low, and can’t be changed? No more! This Viewer does away with the old GPU table and instead uses a quick benchmark measurement to detect your GPU to assign appropriate default graphics settings on startup. The settings on shiny powerful hardware should really let that hardware shine. Get a Project Benchmark Viewer today and help us gather metrics!  Please file bugs in JIRA if you find them.

The new log-in viewer is currently the only release candidate viewer sitting in the viewer release channel. As such, it is liable to be promoted to the de facto release viewer in the near future – probably in week 41 (week commencing Monday October 6th), assuming the statistics for it haven’t shown up any issues.

As the Lab’s blog-post indicates, this viewer is being introduced as a result of several months of A/B testing with the current viewer log-in screen. This testing appears to show that new user retention is some 3-5% better when incoming users are presented with the updated viewer’s log-in / splash screens than when compared with those for the current version.

For those interested in finding out how the new viewer differs from the current version, I have an overview of the new version already posted.

The log-in / splash screen in the login RC viewer seen by users who have previously logged-in to SL

The log-in / splash screen in the login RC viewer seen by users who have previously logged-in to SL

A point to note with the log-in screen changes is that they do not impact the widgets, etc., used by TPVs. Therefore, these changes shouldn’t force those TPVs using their own log-in splash screens to replace them with the Lab’s updates.

The final two aspects of the Lab’s blog post are the deployment of the CDN, which is currently for texture and mesh fetching, and which I’ve also extensively documented through my week SL project updates. At the time of writing, the CDN is available in ten regions across the main grid: Denby, Hippo Hollow, Hippotropolis, Testsylvania, Brasil Rio, Brocade, Fluffy, Freedom City, Rocket City or Whippersnapper. However, more regions will be added as time goes on.

There is no requirement for any special viewer in order to get an idea of the faster downloading of textures and meshes users should witness on entering any of these regions (there may be some rare instances where things are a little slower if you happen to reside closer to one of the Lab’s data centres than to your local CDN node, but these instances are likely to be very rare). However, once the CDN service is available across the grid, it may see a final viewer-side update as a part of final fine-tuning, and well as potentially being extended to include the delivery of other viewer-consumable assets.

The HTTP work, which has been ongoing for the last couple of years and very much a focus of Monty Linden’s work, is something I’ve also reported upon through my weekly SL project updates. This should have some general improvements on performance, both with texture and mesh downloads through the CDN, and with other HTTP-specific SL services. This viewer code is allegedly so fast, the Lab refer to it internally as the “weaponized viewer”.

The benefit of the CDN and the HTTP viewer code – which TPVs are being encouraged to adopt as quickly as their merge / test / release cycles allow – is summed-up in the closing comments on the Lab’s post:

Separately, each of these will improve texture and mesh loading performance, but put together, you should really see some exciting improvements in how long it takes to load new areas and objects – making touring the many fabulous places in Second Life you have not yet visited even better!

Those who have been independently testing both the CDN and the pipelining viewer (in a pre-project viewer release state) have been reporting that results with either / both are impressive. Check Shug Maitland’s comment on this blog, for example, after she tried the CDN regions with a current viewer.

Lab updates corporate leadership page

LL logoI generally keep an eye on the Lab’s corporate website, but confess that things have been such that over the last month, other things have been keeping me occupied so I’ve been a little lax in my checks; however, the Lab have refreshed the Leadership section of the company’s About Page. I’m not sure precisely when this happened, but it appears to have been some time towards the end of August 2014, or early September.

The updated Leadership section of the page sees an expanded management team list complete with photos for all of those on it, rather than the mix of photos and the “creation” images previously found against individual bios.

New to the page (but not necessarily to the Lab) are photos and bios for Rob Anderberg, Senior Director of Development, Pam Beyazit, Senior Director of HR, Scott Reismanis, Director of Digital, and Peter Gray, Director, Global Communications.

LL’s management team: LL’s management team: Rob Anderberg, Pam Beyazit, Scott Reismanis (of Desura fame) and Peter Gray (tow row) join Ebbe Altberg, Kelly Conway, Don Laabs (Danger Linden), Landon McDowell (Landon Linden) and Jeff Petersen (Bagman Linden) (bottom row) on the Lab’s corporate website management page

They all join the familiar line-up of Ebbe Altberg, Kelly Conway, Don Laabs, Landon McDowell and Jeff Petersen.

Gone from the management list is John Laurence, VP of Product, although his LinkedIn bio still records him as working at the Lab (and he was still listed as a member of the management team in August 2014). if he has in fact recently left the Lab, he succeeds Lee Senderov, formerly the Lab’s VP of Marketing, as the most recent departure from the Lab’s management team; Ms Sederov having moved on from the Lab around April 2014 to join Shopular as the Head of Marketing there.

The list of board members remains unchanged since Will Wright’s departure towards the start of 2014.

These updates both reflect changes to the Lab’s management structure and a gradual re-tuning of the corporate website itself, which also saw the removal of the Beta Sign-up option from the menu bar at the top of each page some time around the end of August, and which had previously seen the tag-line “Makers of Shared Creative Spaces” replaced by “Build Worlds With Us” some time in July or August 2014.

A final potential point of interest on the corporate site lies in the Careers Page, which has a list of ongoing career opportunities most likely linked to the Lab’s planned staff expansion to help in the development of their next generation platform. The point of interest is that two of the current positions  – for a Senor Software Engineer and a Senior Software Engineer, Avatar – are referred to as being located in Cambridge, Massachusetts, rather than at the Lab’s Boston office, as one might expect. Does this signify that some of the work on the new platform is being carried out somewhat separately from the Lab’s core activities on the East Coast? Time may tell.

“A ballet in a war zone, beautiful, terrifying, and glorious” – inside LL’s Ops team

secondlifeIn May of 2014, Landon Linden, aka Landon McDowell, the Lab’s VP of Operations and Platform Engineering, wrote a blog post on the reasons why a series of issues combined to make Second Life especially uncomfortable for many.

At the time, and as many bloggers and commentators – myself included – noted, the post came as a refreshing breath of fresh air after so long without meat-and-veg communications from the Lab in terms of what is going on with the platform and why things can go wrong.

Now Landon is back explaining how the Lab’s Ops team responds to issues within their services, the communications tools they use – and why the tools are so effective.

An Inside Look at How The Ops Team Collaborates is once again an interesting and informative piece, delving into not only the technical aspects of how the Lab respond to problems within their services, but which also encompasses the very human aspects of the dealing with issues – handling emotions when tensions are high, opening the window for those not directly involved in matter to keep an eye on what is happening so that they can also make better informed decisions on their own actions, and more.

Landon McDowell, the Lab's VP of Operations and Platform Engineering and his alter-ego, Landon Linden

Landon McDowell, the Lab’s VP of Operations and Platform Engineering and his alter-ego, Landon Linden

The core of the Lab’s approach to incident communications is the use of text chat (specifically IRC) rather than any reliance on crash team meetings, the telephone and so on. Those who deal with the Lab on a technical level won’t be surprised at the use of IRC – it is a fairly strong channel of communication for the Lab in a number of areas; but what makes this post particularly interesting is the manner in which the use of IRC is presented and used: as a central incident and problem management tool for active issues; as a means of ensuring people can quickly get up-to-speed with both what has happened in a situation, and what has been determined / done in trying to deal with it; as a means of providing post-mortem information;  and as a tool for helping train new hires.

These benefits start with what is seen as the sheer speed of communication chat allows, as Landon notes:

The speed of text communication is much faster. The average adult can read about twice as fast as they can listen. This effect is amplified with chat comms being multiplexed, meaning multiple speakers can talk intelligibly at the same time. With practice, a participant can even quickly understand multiple conversations interleaved in the same channel. The power of this cannot be overstated.

In a room or on a conference call, there can only be one speaker at a time. During an outage when tensions are high this kind of order can be difficult to maintain. People naturally want to blurt out what they are seeing. There are methods of dealing with this, such as leader-designating speakers or “conch shell” type protocols. In practice though, what often prevails is what one of my vendors calls the “Mountain View Protocol,” where the loudest speaker is the one who’s heard.

In text, responders are able to hop out of a conversation, focus on some investigation or action, hop back in, and quickly catch up due to the presence of scroll back. In verbal comms, responders check-out to do some work and lose track of the conversation resulting in a lot of repeating.

He also notes that not everyone is involved in a situation right from the start. Issues get escalated as they evolve, additional support may be called-in, or the net widened in the search for underlying causes, requiring additional teams to be involved, or the impact of an incident spreads. Chat and the idea of “reading scrollback” as the Lab calls it, allows people to come on-stream for a given situation and fully au fait with what has occurred and what is happening in a manner not always possible through voice communications and briefings, and without breaking the ongoing flow of communications and thinking on the issue.

The multiplexing capabilities of chat also mean that individuals can disengage from the main conversation, have private exchanges which, while pertinent to the issue, might otherwise derail the core conversation or even be silenced in something like a teleconference – and those engaged in such exchanges can still keep abreast of the central conversations.

For an environment like the Lab, where operations and personnel are distributed (data centres and offices located in different states / on different coasts, not everyone working from an office environment, etc.), chat has proven a powerful tool, although one that may take time getting to grips with, as Landon notes about his first exposure, saying:

I … just sat there staring at the screen wondering what the hell had just happened, wondering what the hell I had gotten myself into. I thought I was a seasoned pro, but I had never ever seen an incident response go that smoothly or quickly. Panic started to set in. I was out of my league.

However, the benefits in using it far outweigh any need for a degree of gear shifting required by ops staff in learning to use the approach. As Landon states in closing his comments, “when it works it is a wondrous thing to behold, a ballet in a war zone, beautiful, terrifying, and glorious.”

This is another great insight into what happens inside the Lab, and as such, the post makes very worthwhile reading, whether or not you have a background in Ops support.

Joe Miller

The news of Joe Miller’s passing has been circulating through the SL community for the last 24 hours, and has included a short tribute to his time with Linden Lab published on the Lab’s blog.

Mr. Miller served as the Lab’s Vice President of Platform and Technology Development from May 2006 through until December 2010, and as such, oversaw many of the key technical developments within Second Life.

Joe Miller's alter-ego at the Lab: Joe Linden, as seen on his Twitter page

Joe Miller’s alter-ego at the Lab: Joe Linden, as seen on his Twitter page

These included such activities as moving the grid away from the traditional “black Wednesday” downtimes while things were being banged upon for around eight hours, and users were faced with having to find something else to do with their online time; overseeing the arrival of voice in Second Life; improving the look of SL with the introduction of Windlight; and improving the overall stability of the viewer.

Throughout all of this, Mr. Miller, through his robotic alter-ego, Joe Linden, was popular among SL users, and not averse to meeting residents and participating in meetings. Via the Lab’s own podcast series, Inside the Lab, he discussed many of the challenges involved in running a service such as Second Life which, as Ciaran Laval (who has my thanks for providing the link) pointed out, can be as pertinent today as they were six years ago.

I didn’t actually get to meet by Joe Miller in-world while he worked at the Lab, but we did seem to share something of a passion for space exploration and astronomy. On Twitter and elsewhere, he would point to stunning astronomy and space images; one in particular that springs to mind is also a favourite of mine – a shot of the Earth and the Moon sitting against a backdrop of stars as captured in 2010 by NASA’s Messenger space vehicle when it was 183 million kilometres away, en route to a rendezvous with Mercury.

Joe Miller, a keen sports fan, joined Sportsvision as VP of Engineering after leaving Linden Lab in 2010

Joe Miller, a keen sports fan, joined Sportvision as VP, Engineering after leaving Linden Lab in 2010 (image courtesy of Sportvision.com)

Outside of his tenure at Linden Lab, joe Miller had a wide-range career in technology spanning some 30 years, and included time at Atari, Convergent Inc., Sega America and SegaSoft Inc. At the latter two, he respectively served as Senior Vice President, Product Development; and Executive Vice President, CTO, and board member.

Alongside of this, he also founded a number of companies and organisations during his career, including the Perilux Group, a product design company, which was engaged to develop several award-winning products now offered by LeapFrog (he is credited as one of the co-inventors of the original Leapster hand-held educational gaming console for young children), Bright Things, Apple, and Fitniks. He also founded the Knowledge Universe Interactive Studio, where he served as both President and CEO as well as serving on the board of directors.

Following his departure from the Lab, he went on to work at Sportvision Inc, as Vice President, Engineering,

Mr. Miller passed away peacefully on July 27th, 2014, with his family at his side. A memorial page has been established by his family where those who know him can remember him and perhaps leave a few words.