Free Museums, Warren Miller, and Birthday Recap

November has been a busy month for Erin and I which is one of the reasons I haven't written recently.  Both of us have been battling a cold which has made me even less excited about writing.  As I turn one year older and feel like I'm getting over this cold I feel like it's time to catch up.  Most of my time has been spent programming a new water gate program for local and international distribution.  Fortunately, Erin and I have found time on the weekends for some fun activities.

On the 7th, Denver museums we're having a free night and shuttles to help people get around.  Erin and I started out at the Museum of Contemporary Art around 5pm.  The art was okay, but the best part was that the bar on the roof was open allowing us to get a drink while looking over the city.  Before long our friends, Brian, Cassie, and Neil joined us on the roof and with the Autumn sun set we watched fireworks being shot from a nearby roof.  We ended up getting some dinner at a pub near Neil's place before heading over to the Molly Brown House Museum.  The main draw was to see my aunt Janet dressed up as Margaret Brown in traditional clothes and a bonnet.  Unfortunately, the line was so long to get in the house that by the time we we're finished and had taken photos the museums were closing so we ended up at a nearby bar before heading home.

The following weekend Erin's school was showing the latest Warren Miller film with a lift ticket to one of the ski resorts included in the movie ticket price.  My friend Andres and his friend Joe joined us for the night as we enjoyed the ski film, vied for fabulous door prizes, and finished the night back at our place playing video games.  It was a fun night, but by the next morning I had finally come down with the cold Erin had been battling for a week already and I wasn't feeling like doing anything but staying home on the couch.

Fortunately by Thursday of this week I was feeling better for my birthday.  Erin and I joined our friends at a supper club that we've been holding for the past few months.  This month found us at the Budapest Bistro on Pearl St.  While the food was a little salty it was good and I enjoyed the change.  I especially enjoyed the chocolate crepe my friend Erich got me.  What was really surprising was when we saw Erin's parents walk by the window and waved them in.  We had a nice chat with them before returning to our meal and being pleasantly surprised when the check came to find that they had paid for Erin and my dinner.  Of course, the best gift of my birthday were tickets to the U2 concert this summer that Erin got us.

We were fortunate to see Erin's parents again on Saturday as we went out to lunch at their favorite Vietnamese restaurant.  It reminded me of going to a great Vietnamese place with my friend Matt back in college when we got a whole fish.  Upon hearing this Erin and her parents insisted that we get the same dish.  The food was delicious and it was funny to watch Erin's dad make the fish talk.

Saturday night was when we had planned to celebrate my birthday with some "shady" activities.  We started the night at the Green Mountain Family Shooting Center.  While Erin was hesitant about shooting she soon got the feel of it and we all had a good time.  We followed shooting with dinner at a nearby Chinese restaurant in a sketchy-looking part of Lakewood.  While the neighborhood doesn't look good the food was delicious and we were soon on to our next stop at a nearby bar with billiards tables.

It felt appropriate for the theme of the night that as we were pulling into the parking lot for the bar we were following two police cars that seemed to be heading for the same place.  In the end the cops just seemed to be making sure nothing bad went on and we had a great time drinking, playing pool, and talking.  All in all I had a great time and was glad my friends could come out and share in the activities.

Snow Daze and Halloween Fun

This past week has been an interesting one as we got our first major snow storm.  What was remarkable was that because of it Erin got, not one, not two, but three snow days where she didn't have to go in to work.  On Wednesday she was absolutely ecstatic and while I was happy for her I wished at the time (6am) that she would stop jumping up and down on the bed so I could get my last hour of sleep.  On Thursday, as the storm worsened and Erin learned the night before that there wouldn't be school for a second day we were both happy.  By Friday though, as Erin learned that she would have a third snow day, her feelings were mixed.  Having been cooped up inside for the past three days we were starting to get a little stir crazy.  Erin's also worried now that she'll have to make up some of those days on Saturdays or in the summer.  Still, I'm happy for the extra time we got to spend with each other (even if I still had to do my work).

By the time Halloween came on Saturday night we were excited to get dressed up and head out.  We were having dinner and doing pumpkin carving at our friend's Rachel and Cameron's house.  The party was a lot of fun as we had some delicious chilli and enjoyed carving our pumpkin.  While our pumpkin was very soft and kind of frozen from sitting out in the snow storm we were still able to carve a decent Frankenstein monster in it.  By the end of the night we were exhausted though and happy to sleep in Rachel and Cameron's guest bedroom. 

 For our costumes Erin had seen a Halloween costume news piece with a picture of a guy with a golf ball sticking out of his head so we re-created the costumes of an unfortunate golfing accident.  The great part is that we were able to get most of our costumes at a Goodwill store for very little money.  Of course as we were taking the photo Erin asked, "So how do I hold this golf club?"...It may be awhile before I take her golfing.

Fall Break

As Fall Break ends for Erin we're reminiscing on all that we've done over the past week.  In fact, it's been a few weeks since I've written.  There were some high points since then such as our supper club at "Bones" in Denver and dinner at our friend's Rachel and Cameron's house followed by drinks with our friends Pete, Leah, Erich, and Brad at the Candlelight.  But much of the time was spent working as Erin has been trying to keep up with grading and I've been programming.

That's why this week has been so fun as we've taken time to enjoy some free time together.  The break got off to a shaky start as Erin pulled a muscle in her neck last weekend and needed to not move too much for a couple of days.  But on Monday she was ready to go as we drove down to Penrose for some fall-time harvesting.  We had found an apple orchard down there that also had a pumpkin patch.  It was fun to get out and do some picking.  It reminded me a lot about picking apricots from the tree in our backyard when I was growing up.  While at the orchard we were also able to pick out a nice big pumpkin.  It ended up being a fun time even though it included a lot of driving in comparison to picking.

On Tuesday we left on a surprise that Erin had planned for me.  We ended up on a beautiful hike above Boulder before the weather turned bad.  Then Erin continued the surprise by checking us into the Bradley Bed and Breakfast in Boulder.  It turned out to be one of the best B&B's that we've stayed in and was quite romantic.  To top it all off we went to a wonderful French resteraunt on Pearl St. called L'Atelier.  While we had been planning on going on another hike on Wednesday the snow that had come over night and the continuing rain drove us back to Castle Rock.

Thursday night is was my turn to plan the evening.  Based on a movie we had watched the previous weekend we decided to have a "James Bond" night.  We got dressed up and went out on the town (Castle Rock) to find the best martinis.  While the first place we went to, Uniscali, didn't have martinis, they did have some very good appetizers that served as dinner.  Our next stop, Sienna's, proved to be better for our evening with $5 martini happy hour and a wide selection.  After a quick stop at the video store we finished the evening back at home, curled up in front of the fire watching another cheesy Bond film.

On Friday night we came to the part of the week that we had been looking forward to for a long time.  Back in April Erin had gotten tickets for us to go see "Wicked", a musical based on what happens in Oz before Dorothy gets there in "The Wizard of Oz."  It is a very interesting and compelling story that I enjoyed very much.  Having seen it once before, Erin was convinced that if I was going to like a musical, this would be it.  We both had a great time at the show and I would see it again.  We had also decided that we would make the night extra special by staying for the night at the hotel Magnolia.  We had stayed there once before and found it very nice for the price.

On Saturday our adventure continued as we went to Littleton for a walk along the Platt river and lunch at Tres Jolie.  My aunt and uncle had gotten us a gift certificate for the shop and it was very nice.  The teas were delicious as were the pastries that accompanied them.  Having some time we then went in search of Halloween costumes.  Our first stop was the popular Reinke Bros. in downtown Littleton.  While they had several traditional costumes and accessories we were narrowing in on something a bit more creative.  A stop at the nearby Goodwill store provided the clothes we needed for our costumes.  I'll leave the surprise of what we're going to be for my next post.

To finish off Saturday, Erin went out with her friends to "Canvas and Cocktails" where they all enjoyed drinks while painting.  At the same time I met up with my friend Andres to play some video games and just hang out.  It turned out to be a fun night for both Erin and I. 

So today as the Fall Break comes to an end it has been nice to look back at all the fun things we've done.  We're going to catch a movie with our friends Brian and Cassie this afternoon before descending into the reality of the early wake-up and return to work that comes tomorrow.  Still, there's always next weekend to look forward to!

NeuroEvolution

I've finally got a working neuroevolution base which is doing well with basic conditional tests (e.g. xor). Now there are all sorts of avenues I want to start exploring and I'm having a hard time deciding which to follow:

1. Node functions: Currently, each node utilizes a step function based on a random threshold. Most neural networks use some combination of sigmoid and similar functions, so I'd like my neural networks to have use of those as well. However, I then saw a paper on Cartesian Genetic Programming (CGP) which unlike genetic programming (GP) uses a network structure (instead of a tree) to represent programs and emphasizes separating the genotype and the phenotypic representation. As I read this I thought about giving the nodes in my networks even more functions to use: how could I have functions that take in types that are not real numbers? How would nodes coordinate between their outputs and expected inputs if they're different types?


2. Evolutionary Strategies: There seems to be some different definitions for this, but I take it to mean applying a genetic algorithm not just to the neural networks I'm developing, but also to the parameters that define the bounds on those networks. In particular, there's a method called Covariance Matrix Adaptation - Evolutionary Strategy (CMA-ES) which drives these changes in a way that reduces wasting effort on parameters that are closely correlated.


3. Problem Classification: Currently my networks can handle an array of real numbers for their input and produce an array of real numbers for their output. I'd like the program to handle different forms of data and produce different types of output. What's more, once the program has solved several different types of problems, it should be able to tell from the data coming in (based on a classifier network) what kind of problem it is and create the initial parameters accordingly.


4. Human-Assisted Selection: There are already several systems that create a random group of individuals, ask the user to select which ones they prefer, and make the next generation from that. I'd like to have the same ability with my program. On another level, a neural network could use the initial population of individuals as its input and the user's selection as its expected output to train it in making suggestions to the user.


5. Multilevel Selection: By creating additional populations of individuals, more complex behavior can be generated...

In reality, all of these are fascinating concepts that I would like to explore, but the choice of problem is the problem. All of these avenues of interest are things that I believe would make the program smarter, but how do you prove it without a good test. There are common tests like the double pole balancing problem which I'm incorporating, but where do I go from there?

It has started me thinking on a very interesting set of questions: Can you use neuroevolution in a cumulative fashion so that you train a network to do one thing and then train it to do something more advanced based on the initial training? It's the adage that you have to crawl before you can walk, and walk before you can run (even though one isn't use to do the next).

Are there learning tracks where you train a neural network the first thing on the track and that prepares it to learn the next thing, and so on? Or is it more of a network where each learning outcome is a node and the nodes feeding into the next layer of nodes all need to be learned before the next layer can be learned? This seems more logical to me. Not only does a person need to learn how to crawl, but they also must learn how to stand before they learn how to walk. Maybe a feed-forward network, since it's unlikely that something learned down the line would be required for an earlier learning node (though back-propagation does seem in effect to strengthen the earlier understandings).

The question then becomes, what starts the whole thing off? What are the first things that babies learn and must master before they can progress to the next stage of learning? Certainly as a person gains experience in multiple, cross-cutting fields (e.g. music, language), the inspiration and understanding to progress becomes more complex, but what about at the very beginning? Is it something that can be mapped out? Has someone already done this?

Where Did Summer Go?

Or even Autumn for that matter?  One week it's summer and the next it feels like we skipped directly to Winter.  The picture to the left is from our porch in Castle Rock, CO where it has started snowing.  We had snow a week or two ago too, but this is the first time it's actually stayed on the ground.  I didn't think this was supposed to happen at least until Halloween!

I suppose it was a sign of the season as Erin and I went to get our flu shots on Tuesday last week.  While our arms both felt sore afterward, we weren't otherwise side-affected.

Still, when my friend Andres invited me to watch baseball last Thursday I thought it was a good omen for the last remnants of Summer.  While the game was a lot of fun and the Rockies won, I was sorely disappointed in terms of the weather.  It was okay in the sun, but the wind was bitterly cold in the shade where we were seated.  It was nice to get after the game and get a beer and a burger.

Then this past weekend Erin and I attempted to ride up to see the Fall foliage like we did last year (Enjoying Fall Foliage, Ouzo, and Bowling).  This time we were able to go with our friends Brian and Cassie (also on their motorcycle).  After meeting our wedding videographer to see a first cut of our wedding video (which looked very nice) we met up and headed up to the mountains.  The ride started nicely, but got colder the higher up we got until we stopped at a light and Erin exclaimed how cold she was.  Still, I pressed on hoping to find a good spot to view the changing colors.  After a while though I realized that it wasn't going to happen with the route we had chosen and the approaching storm clouds.  We turned around and went back to Brian and Cassie's place.

Back in the warmth of their home we were treated to a warm, delicious meal and a touching movie.  We also got to spend time with Neal's dog Bart (pictured right) who Brian and Cassie were looking after for the weekend.  He's such a friendly and well-trained dog that it's hard not to like him.

On Sunday Erin and I did some work and ran errands until it was time to start preparing dinner.  It was our pleasure to host my aunt and uncle down to our place for some lemon chicken picatta.  We had a great time chatting and catching up.

So now Erin and I are getting ready to head out to our monthly supper club up in Denver.  Of course it would be snowing and I hope it lets up soon.  Fortunately Erin doesn't have work tomorrow so we can sleep in, stay inside, and hopefully keep warm!

Wisconsin

This past weekend Erin and I flew out to Milwaukee to celebrate one of her cousin's getting married.  As I had never been to Wisconsin before I was interested to see what it would be like.  So on Friday morning we got up early and got to the airport with no problems.  Arriving in Milwaukee in the afternoon we met up with Erin's parents who had rented a car.  Driving through the city on our way out to the city of Brown Deer where our hotel was provided a nice view of the city.  Much of the countryside reminded me of Georgia for some reason with its dense woods and lush green lawns.

At our hotel we met up with Erin's brother, Andrew, and his girlfriend, Sara.  Once we had all checked in we found that the hotel bar was serving martinis and decided to have a drink.  After finishing our drinks we thought it would be good if we went out in search of a walk to do before the rehearsal dinner.  While we were able to find some nice river-side roads near our hotel it soon began rain and walking was out of the question.  So we returned to the hotel bar for more to drink as we played cards and waited for dinner.  The dinner was a delicious Italian meal out at the brides' parent's house where I got to meet much of Erin's extended family.  They were all very nice and we had a great time.  We even had a chance to sit around the fire pit as the rain let up.

On Saturday we got up and had breakfast at a nearby Denny's before heading out to walk around a lake we had driven by the day before.  The area was very nice and green, but with the sky still overcast and the humidity starting to be felt more heavily, it seemed to be a very wet area.  After getting cleaned up we were off to the traditional Catholic wedding just outside of historic Cedarsburg.  Afterwards, as pictures were being taken we had time to wander around the charming city before the reception.  One of our most favorite places was a brew pub down on the river.  In a very relaxing setting Erin finally found a beer that she liked (Framboise) and we all enjoyed the chance to talk.  As the reception was being held at the hotel we were staying at we eventually made our way back there for the festivities.  The new couple looked very happy together as they danced and we got a chance to see a slide-show of pictures from their lives.

For Sunday we had talked about several potential things to do before we all left.  We started out with some geocaching, using a GPS to find a box hidden by others.  While it took us a while to find the cache it turned out to be in a very pretty nature preserve.  We were then on our way back to city to see the Andy Warhol exhibit in the Milwaukee Art Museum.  The museum had some very nice exhibits and some exciting architecture.  It includes an elaborate, wing-like brise soleil which shades the museum (pictured as its opening).

Before too long though it was time to catch our flight.  We said our goodbyes and made out way to the airport.  It was good visiting with all of Erin's relatives and getting to see Wisconsin.  I certainly have a higher opinion of the area now that I've had a chance to see it first-hand.

Kart Racing, IMAX, and Pottery Painting

This past weekend was an exciting one for Erin and I as we did a diversity of activities.  Friday, after Erin got off work, we went over to the Union for happy hour - which is turning out to be a low-cost ritual for our Friday afternoons.  That evening we played a board game while eating apple pie and drinking port.  It was a very enjoyable experience that Erin had set up and shared with me.

On Saturday we went to celebrate our friend Andres' birthday out at the Centennial Track.  There we all got suited up with helmets, gloves, and overalls before climbing into karts for some racing around the .8 mile track.  While we only got to race for about 10 minutes it was fun.  After the race we hung out a bit with Andres and the rest of his friends before going over to Erin's parent's house for dinner.

Erin's sister Amy was in town preparing to defend her thesis on Monday, so we went to see her and visit with Erin's parents.  They took us all out to this authentic Mexican restaurant called "El Paraiso" in North-West Denver.  Unlike the typical tacos and burritos this place had a wide variety of traditional Mexican dishes.  The food turned out to be good and with generous portions.  There was also a mariachi band playing adding to the atmosphere.

More fun came later in the weekend as Erin and I went downtown to the Denver Museum of Nature and Science to watch an IMAX show.  Being located in city park, the museum's location gave us a chance to have a nice walk around the park before our show started.  We watched a show called "Wild Ocean" about an aquatic feeding frenzy that happens off the coast of South Africa each year.  It was amazing being immersed in the underwater action the way that an IMAX movie provides.

After the show we went to Ceramics in the City where we got to pick out a piece of pottery to paint.  After a long deliberation we settled on salt and pepper shakers along with a common theme of nature.  We both had a really good time as we painted our designs on our shakers.  We finally finished and gave the shakers to the attendant to be glazed and fired.  We're looking forward to seeing how they turn out next week.