Christmas Tales

Well with Christmas over and the holidays winding down I thought it would be a good time to reflect on the wonderful events that Erin and I have gotten to be a part of over the past few weeks.

The festivities began about two weeks ago when we went to our supper club, where each month one couple from the group picks a new place for us to get together for dinner.  This last time we all met at Tambien (near Cherry Creek Mall) for Mexican food as well as a White Elephant gift exchange.  If you can't tell from the picture I got an exciting leopard-print snuggie while Erin ended up with one of the gifts we had given.  It was all a lot of fun as we enjoyed some good food.

Later that week, Erin's sister Amy was in town to receive her PhD in Atmospheric Science from CSU.  The whole family was very proud of her and there were some good speeches at the graduation ceremony.  It was also nice to see Fort Collins and have dinner up there.

On Tuesday, last week, Erin and I made our way to Littleton for dinner with my aunt and uncle.  They served us a delicious chili meal before we did our gift exchange.  They gave us several wine-related gifts as they knew we both appreciated wine, including at very nice picnic wine case.  I can see us getting a lot of use out of it.  Erin and I also had a lot of fun designing some ceramic salt and pepper shakers for them with activities that my aunt and uncle enjoy.

On Christmas eve Erin and I went to spend time with Erin's family.  We started by all meeting up at the movie theater to watch "Up in the Air" with George Clooney.  It was an interesting movie about a life of permanent travel and the lack of personal relationships that usually entails.  After the movie we went out for a delicious lunch.  The rest of the evening was spent preparing and eating dinner, playing a rivoting game of darts, and just enjoying each-others company.

Christmas day found us opening presents and stockings.  Erin now has several books to read for the rest of her Winter break this upcoming week and I have a gift certificate to get some new clothes.  Erin and I spent much of the rest of the day putting together a puzzle, but we took breaks to play games such as one Amy's husband Kevin came up with which involves writing a phrase at the top of a piece of paper and passing it to the person on your left.  The fun in the game comes from having to draw a picture of the phrase, folding the paper so the next person can't see the original phrase, passing it to the next person, and writing a description of the picture you've been handed.  As this continues the phrases and the pictures start to look nothing like the original phrase and once the paper is full, it's fun to look back at how things changed.

On the day after Christmas some of Erin's parents' friends came by for breakfast and there was a lot of catching up.  Then Amy and Kevin came back by before Amy had to head back to the airport.  We enjoyed lunch and a walk at the Streets of Southglen before packing up and heading back home.  All in all it's been a very nice Christmas and Erin and I are very grateful for the wonderful friends and family that we've had a chance to spend it with.

Thanksgiving, Weird Musical, and Lots of Work

I've been very busy with work these past few weeks and have let blogging fall behind, but I'm having a hard time focusing on work this morning so I thought it was time to catch up here. 

First off, Erin and I had an enjoyable trip out to California for Thanksgiving.  While there were short lines at the airport and a comfortable flight over, our smooth traveling came to an end when we got out onto the 405 freeway in Los Angeles.  I knew it was going to be bad from previous experience, but I guess I hadn't considered how bad when we were making our plans and buying tickets.  Six hours of driving later we finally made it to my parents place in Tulare.  The next day, Thanksgiving, was clear and warm out so we enjoyed going on a walk and sitting out in the sun before having a delicious meal.  Friday night gave us a chance to see our friends Joe and Angie before heading back home on Saturday.  As the trip back took far less time we still had the energy when we got back to Denver to go out and celebrate two years of being together at Nono's Cafe where we had our first date.

The main excitement last week came when someone at Erin's school was giving away tickets to the musical "Spring Awakening".  Erin had been interested in seeing the musical mainly because the theater group had sent out a rare warning on the potential shocking nature of the play.  That and the fact that Erin was able to get the well-placed seat tickets for free enticed me to join her.  The play did deliver on the shocking promise of brief nudity, simulated teenage sex, and other adult themes including rape and masochism.  However, that didn't bother me as much as the singing which didn't seem to always fit with the original play and the expressionist nature of bright lights shining on the audience.  The on-stage band's musical performance though was often very good and the whole thing was a unique experience.  Still, I don't think I'll be attending many more musicals in the near future.

The highlight of this week has been picking Erin's parents up at the airport after their exciting trip to Argentina.  Unfortunately, far too much of that excitement came from being mugged in Buenos Aires at the end of their trip.  Fortunately both of them came through the experience with only large scrapes and some money gone.  Erin's mom even got her snatched purse back after visiting the police station in the rough neighborhood where the attack happened.  Erin and I are glad that they made it back home safe and sound.

Well, I guess I had better get back to work now.  But I'll leave with a picture of the little Christmas tree that we decorated Wednesday night.

Free Museums, Warren Miller, and Birthday Recap

November has been a busy month for Erin and I which is one of the reasons I haven't written recently.  Both of us have been battling a cold which has made me even less excited about writing.  As I turn one year older and feel like I'm getting over this cold I feel like it's time to catch up.  Most of my time has been spent programming a new water gate program for local and international distribution.  Fortunately, Erin and I have found time on the weekends for some fun activities.

On the 7th, Denver museums we're having a free night and shuttles to help people get around.  Erin and I started out at the Museum of Contemporary Art around 5pm.  The art was okay, but the best part was that the bar on the roof was open allowing us to get a drink while looking over the city.  Before long our friends, Brian, Cassie, and Neil joined us on the roof and with the Autumn sun set we watched fireworks being shot from a nearby roof.  We ended up getting some dinner at a pub near Neil's place before heading over to the Molly Brown House Museum.  The main draw was to see my aunt Janet dressed up as Margaret Brown in traditional clothes and a bonnet.  Unfortunately, the line was so long to get in the house that by the time we we're finished and had taken photos the museums were closing so we ended up at a nearby bar before heading home.

The following weekend Erin's school was showing the latest Warren Miller film with a lift ticket to one of the ski resorts included in the movie ticket price.  My friend Andres and his friend Joe joined us for the night as we enjoyed the ski film, vied for fabulous door prizes, and finished the night back at our place playing video games.  It was a fun night, but by the next morning I had finally come down with the cold Erin had been battling for a week already and I wasn't feeling like doing anything but staying home on the couch.

Fortunately by Thursday of this week I was feeling better for my birthday.  Erin and I joined our friends at a supper club that we've been holding for the past few months.  This month found us at the Budapest Bistro on Pearl St.  While the food was a little salty it was good and I enjoyed the change.  I especially enjoyed the chocolate crepe my friend Erich got me.  What was really surprising was when we saw Erin's parents walk by the window and waved them in.  We had a nice chat with them before returning to our meal and being pleasantly surprised when the check came to find that they had paid for Erin and my dinner.  Of course, the best gift of my birthday were tickets to the U2 concert this summer that Erin got us.

We were fortunate to see Erin's parents again on Saturday as we went out to lunch at their favorite Vietnamese restaurant.  It reminded me of going to a great Vietnamese place with my friend Matt back in college when we got a whole fish.  Upon hearing this Erin and her parents insisted that we get the same dish.  The food was delicious and it was funny to watch Erin's dad make the fish talk.

Saturday night was when we had planned to celebrate my birthday with some "shady" activities.  We started the night at the Green Mountain Family Shooting Center.  While Erin was hesitant about shooting she soon got the feel of it and we all had a good time.  We followed shooting with dinner at a nearby Chinese restaurant in a sketchy-looking part of Lakewood.  While the neighborhood doesn't look good the food was delicious and we were soon on to our next stop at a nearby bar with billiards tables.

It felt appropriate for the theme of the night that as we were pulling into the parking lot for the bar we were following two police cars that seemed to be heading for the same place.  In the end the cops just seemed to be making sure nothing bad went on and we had a great time drinking, playing pool, and talking.  All in all I had a great time and was glad my friends could come out and share in the activities.

Snow Daze and Halloween Fun

This past week has been an interesting one as we got our first major snow storm.  What was remarkable was that because of it Erin got, not one, not two, but three snow days where she didn't have to go in to work.  On Wednesday she was absolutely ecstatic and while I was happy for her I wished at the time (6am) that she would stop jumping up and down on the bed so I could get my last hour of sleep.  On Thursday, as the storm worsened and Erin learned the night before that there wouldn't be school for a second day we were both happy.  By Friday though, as Erin learned that she would have a third snow day, her feelings were mixed.  Having been cooped up inside for the past three days we were starting to get a little stir crazy.  Erin's also worried now that she'll have to make up some of those days on Saturdays or in the summer.  Still, I'm happy for the extra time we got to spend with each other (even if I still had to do my work).

By the time Halloween came on Saturday night we were excited to get dressed up and head out.  We were having dinner and doing pumpkin carving at our friend's Rachel and Cameron's house.  The party was a lot of fun as we had some delicious chilli and enjoyed carving our pumpkin.  While our pumpkin was very soft and kind of frozen from sitting out in the snow storm we were still able to carve a decent Frankenstein monster in it.  By the end of the night we were exhausted though and happy to sleep in Rachel and Cameron's guest bedroom. 

 For our costumes Erin had seen a Halloween costume news piece with a picture of a guy with a golf ball sticking out of his head so we re-created the costumes of an unfortunate golfing accident.  The great part is that we were able to get most of our costumes at a Goodwill store for very little money.  Of course as we were taking the photo Erin asked, "So how do I hold this golf club?"...It may be awhile before I take her golfing.

Fall Break

As Fall Break ends for Erin we're reminiscing on all that we've done over the past week.  In fact, it's been a few weeks since I've written.  There were some high points since then such as our supper club at "Bones" in Denver and dinner at our friend's Rachel and Cameron's house followed by drinks with our friends Pete, Leah, Erich, and Brad at the Candlelight.  But much of the time was spent working as Erin has been trying to keep up with grading and I've been programming.

That's why this week has been so fun as we've taken time to enjoy some free time together.  The break got off to a shaky start as Erin pulled a muscle in her neck last weekend and needed to not move too much for a couple of days.  But on Monday she was ready to go as we drove down to Penrose for some fall-time harvesting.  We had found an apple orchard down there that also had a pumpkin patch.  It was fun to get out and do some picking.  It reminded me a lot about picking apricots from the tree in our backyard when I was growing up.  While at the orchard we were also able to pick out a nice big pumpkin.  It ended up being a fun time even though it included a lot of driving in comparison to picking.

On Tuesday we left on a surprise that Erin had planned for me.  We ended up on a beautiful hike above Boulder before the weather turned bad.  Then Erin continued the surprise by checking us into the Bradley Bed and Breakfast in Boulder.  It turned out to be one of the best B&B's that we've stayed in and was quite romantic.  To top it all off we went to a wonderful French resteraunt on Pearl St. called L'Atelier.  While we had been planning on going on another hike on Wednesday the snow that had come over night and the continuing rain drove us back to Castle Rock.

Thursday night is was my turn to plan the evening.  Based on a movie we had watched the previous weekend we decided to have a "James Bond" night.  We got dressed up and went out on the town (Castle Rock) to find the best martinis.  While the first place we went to, Uniscali, didn't have martinis, they did have some very good appetizers that served as dinner.  Our next stop, Sienna's, proved to be better for our evening with $5 martini happy hour and a wide selection.  After a quick stop at the video store we finished the evening back at home, curled up in front of the fire watching another cheesy Bond film.

On Friday night we came to the part of the week that we had been looking forward to for a long time.  Back in April Erin had gotten tickets for us to go see "Wicked", a musical based on what happens in Oz before Dorothy gets there in "The Wizard of Oz."  It is a very interesting and compelling story that I enjoyed very much.  Having seen it once before, Erin was convinced that if I was going to like a musical, this would be it.  We both had a great time at the show and I would see it again.  We had also decided that we would make the night extra special by staying for the night at the hotel Magnolia.  We had stayed there once before and found it very nice for the price.

On Saturday our adventure continued as we went to Littleton for a walk along the Platt river and lunch at Tres Jolie.  My aunt and uncle had gotten us a gift certificate for the shop and it was very nice.  The teas were delicious as were the pastries that accompanied them.  Having some time we then went in search of Halloween costumes.  Our first stop was the popular Reinke Bros. in downtown Littleton.  While they had several traditional costumes and accessories we were narrowing in on something a bit more creative.  A stop at the nearby Goodwill store provided the clothes we needed for our costumes.  I'll leave the surprise of what we're going to be for my next post.

To finish off Saturday, Erin went out with her friends to "Canvas and Cocktails" where they all enjoyed drinks while painting.  At the same time I met up with my friend Andres to play some video games and just hang out.  It turned out to be a fun night for both Erin and I. 

So today as the Fall Break comes to an end it has been nice to look back at all the fun things we've done.  We're going to catch a movie with our friends Brian and Cassie this afternoon before descending into the reality of the early wake-up and return to work that comes tomorrow.  Still, there's always next weekend to look forward to!

NeuroEvolution

I've finally got a working neuroevolution base which is doing well with basic conditional tests (e.g. xor). Now there are all sorts of avenues I want to start exploring and I'm having a hard time deciding which to follow:

1. Node functions: Currently, each node utilizes a step function based on a random threshold. Most neural networks use some combination of sigmoid and similar functions, so I'd like my neural networks to have use of those as well. However, I then saw a paper on Cartesian Genetic Programming (CGP) which unlike genetic programming (GP) uses a network structure (instead of a tree) to represent programs and emphasizes separating the genotype and the phenotypic representation. As I read this I thought about giving the nodes in my networks even more functions to use: how could I have functions that take in types that are not real numbers? How would nodes coordinate between their outputs and expected inputs if they're different types?


2. Evolutionary Strategies: There seems to be some different definitions for this, but I take it to mean applying a genetic algorithm not just to the neural networks I'm developing, but also to the parameters that define the bounds on those networks. In particular, there's a method called Covariance Matrix Adaptation - Evolutionary Strategy (CMA-ES) which drives these changes in a way that reduces wasting effort on parameters that are closely correlated.


3. Problem Classification: Currently my networks can handle an array of real numbers for their input and produce an array of real numbers for their output. I'd like the program to handle different forms of data and produce different types of output. What's more, once the program has solved several different types of problems, it should be able to tell from the data coming in (based on a classifier network) what kind of problem it is and create the initial parameters accordingly.


4. Human-Assisted Selection: There are already several systems that create a random group of individuals, ask the user to select which ones they prefer, and make the next generation from that. I'd like to have the same ability with my program. On another level, a neural network could use the initial population of individuals as its input and the user's selection as its expected output to train it in making suggestions to the user.


5. Multilevel Selection: By creating additional populations of individuals, more complex behavior can be generated...

In reality, all of these are fascinating concepts that I would like to explore, but the choice of problem is the problem. All of these avenues of interest are things that I believe would make the program smarter, but how do you prove it without a good test. There are common tests like the double pole balancing problem which I'm incorporating, but where do I go from there?

It has started me thinking on a very interesting set of questions: Can you use neuroevolution in a cumulative fashion so that you train a network to do one thing and then train it to do something more advanced based on the initial training? It's the adage that you have to crawl before you can walk, and walk before you can run (even though one isn't use to do the next).

Are there learning tracks where you train a neural network the first thing on the track and that prepares it to learn the next thing, and so on? Or is it more of a network where each learning outcome is a node and the nodes feeding into the next layer of nodes all need to be learned before the next layer can be learned? This seems more logical to me. Not only does a person need to learn how to crawl, but they also must learn how to stand before they learn how to walk. Maybe a feed-forward network, since it's unlikely that something learned down the line would be required for an earlier learning node (though back-propagation does seem in effect to strengthen the earlier understandings).

The question then becomes, what starts the whole thing off? What are the first things that babies learn and must master before they can progress to the next stage of learning? Certainly as a person gains experience in multiple, cross-cutting fields (e.g. music, language), the inspiration and understanding to progress becomes more complex, but what about at the very beginning? Is it something that can be mapped out? Has someone already done this?

Where Did Summer Go?

Or even Autumn for that matter?  One week it's summer and the next it feels like we skipped directly to Winter.  The picture to the left is from our porch in Castle Rock, CO where it has started snowing.  We had snow a week or two ago too, but this is the first time it's actually stayed on the ground.  I didn't think this was supposed to happen at least until Halloween!

I suppose it was a sign of the season as Erin and I went to get our flu shots on Tuesday last week.  While our arms both felt sore afterward, we weren't otherwise side-affected.

Still, when my friend Andres invited me to watch baseball last Thursday I thought it was a good omen for the last remnants of Summer.  While the game was a lot of fun and the Rockies won, I was sorely disappointed in terms of the weather.  It was okay in the sun, but the wind was bitterly cold in the shade where we were seated.  It was nice to get after the game and get a beer and a burger.

Then this past weekend Erin and I attempted to ride up to see the Fall foliage like we did last year (Enjoying Fall Foliage, Ouzo, and Bowling).  This time we were able to go with our friends Brian and Cassie (also on their motorcycle).  After meeting our wedding videographer to see a first cut of our wedding video (which looked very nice) we met up and headed up to the mountains.  The ride started nicely, but got colder the higher up we got until we stopped at a light and Erin exclaimed how cold she was.  Still, I pressed on hoping to find a good spot to view the changing colors.  After a while though I realized that it wasn't going to happen with the route we had chosen and the approaching storm clouds.  We turned around and went back to Brian and Cassie's place.

Back in the warmth of their home we were treated to a warm, delicious meal and a touching movie.  We also got to spend time with Neal's dog Bart (pictured right) who Brian and Cassie were looking after for the weekend.  He's such a friendly and well-trained dog that it's hard not to like him.

On Sunday Erin and I did some work and ran errands until it was time to start preparing dinner.  It was our pleasure to host my aunt and uncle down to our place for some lemon chicken picatta.  We had a great time chatting and catching up.

So now Erin and I are getting ready to head out to our monthly supper club up in Denver.  Of course it would be snowing and I hope it lets up soon.  Fortunately Erin doesn't have work tomorrow so we can sleep in, stay inside, and hopefully keep warm!