Fall Break

As Fall Break ends for Erin we're reminiscing on all that we've done over the past week.  In fact, it's been a few weeks since I've written.  There were some high points since then such as our supper club at "Bones" in Denver and dinner at our friend's Rachel and Cameron's house followed by drinks with our friends Pete, Leah, Erich, and Brad at the Candlelight.  But much of the time was spent working as Erin has been trying to keep up with grading and I've been programming.

That's why this week has been so fun as we've taken time to enjoy some free time together.  The break got off to a shaky start as Erin pulled a muscle in her neck last weekend and needed to not move too much for a couple of days.  But on Monday she was ready to go as we drove down to Penrose for some fall-time harvesting.  We had found an apple orchard down there that also had a pumpkin patch.  It was fun to get out and do some picking.  It reminded me a lot about picking apricots from the tree in our backyard when I was growing up.  While at the orchard we were also able to pick out a nice big pumpkin.  It ended up being a fun time even though it included a lot of driving in comparison to picking.

On Tuesday we left on a surprise that Erin had planned for me.  We ended up on a beautiful hike above Boulder before the weather turned bad.  Then Erin continued the surprise by checking us into the Bradley Bed and Breakfast in Boulder.  It turned out to be one of the best B&B's that we've stayed in and was quite romantic.  To top it all off we went to a wonderful French resteraunt on Pearl St. called L'Atelier.  While we had been planning on going on another hike on Wednesday the snow that had come over night and the continuing rain drove us back to Castle Rock.

Thursday night is was my turn to plan the evening.  Based on a movie we had watched the previous weekend we decided to have a "James Bond" night.  We got dressed up and went out on the town (Castle Rock) to find the best martinis.  While the first place we went to, Uniscali, didn't have martinis, they did have some very good appetizers that served as dinner.  Our next stop, Sienna's, proved to be better for our evening with $5 martini happy hour and a wide selection.  After a quick stop at the video store we finished the evening back at home, curled up in front of the fire watching another cheesy Bond film.

On Friday night we came to the part of the week that we had been looking forward to for a long time.  Back in April Erin had gotten tickets for us to go see "Wicked", a musical based on what happens in Oz before Dorothy gets there in "The Wizard of Oz."  It is a very interesting and compelling story that I enjoyed very much.  Having seen it once before, Erin was convinced that if I was going to like a musical, this would be it.  We both had a great time at the show and I would see it again.  We had also decided that we would make the night extra special by staying for the night at the hotel Magnolia.  We had stayed there once before and found it very nice for the price.

On Saturday our adventure continued as we went to Littleton for a walk along the Platt river and lunch at Tres Jolie.  My aunt and uncle had gotten us a gift certificate for the shop and it was very nice.  The teas were delicious as were the pastries that accompanied them.  Having some time we then went in search of Halloween costumes.  Our first stop was the popular Reinke Bros. in downtown Littleton.  While they had several traditional costumes and accessories we were narrowing in on something a bit more creative.  A stop at the nearby Goodwill store provided the clothes we needed for our costumes.  I'll leave the surprise of what we're going to be for my next post.

To finish off Saturday, Erin went out with her friends to "Canvas and Cocktails" where they all enjoyed drinks while painting.  At the same time I met up with my friend Andres to play some video games and just hang out.  It turned out to be a fun night for both Erin and I. 

So today as the Fall Break comes to an end it has been nice to look back at all the fun things we've done.  We're going to catch a movie with our friends Brian and Cassie this afternoon before descending into the reality of the early wake-up and return to work that comes tomorrow.  Still, there's always next weekend to look forward to!

NeuroEvolution

I've finally got a working neuroevolution base which is doing well with basic conditional tests (e.g. xor). Now there are all sorts of avenues I want to start exploring and I'm having a hard time deciding which to follow:

1. Node functions: Currently, each node utilizes a step function based on a random threshold. Most neural networks use some combination of sigmoid and similar functions, so I'd like my neural networks to have use of those as well. However, I then saw a paper on Cartesian Genetic Programming (CGP) which unlike genetic programming (GP) uses a network structure (instead of a tree) to represent programs and emphasizes separating the genotype and the phenotypic representation. As I read this I thought about giving the nodes in my networks even more functions to use: how could I have functions that take in types that are not real numbers? How would nodes coordinate between their outputs and expected inputs if they're different types?


2. Evolutionary Strategies: There seems to be some different definitions for this, but I take it to mean applying a genetic algorithm not just to the neural networks I'm developing, but also to the parameters that define the bounds on those networks. In particular, there's a method called Covariance Matrix Adaptation - Evolutionary Strategy (CMA-ES) which drives these changes in a way that reduces wasting effort on parameters that are closely correlated.


3. Problem Classification: Currently my networks can handle an array of real numbers for their input and produce an array of real numbers for their output. I'd like the program to handle different forms of data and produce different types of output. What's more, once the program has solved several different types of problems, it should be able to tell from the data coming in (based on a classifier network) what kind of problem it is and create the initial parameters accordingly.


4. Human-Assisted Selection: There are already several systems that create a random group of individuals, ask the user to select which ones they prefer, and make the next generation from that. I'd like to have the same ability with my program. On another level, a neural network could use the initial population of individuals as its input and the user's selection as its expected output to train it in making suggestions to the user.


5. Multilevel Selection: By creating additional populations of individuals, more complex behavior can be generated...

In reality, all of these are fascinating concepts that I would like to explore, but the choice of problem is the problem. All of these avenues of interest are things that I believe would make the program smarter, but how do you prove it without a good test. There are common tests like the double pole balancing problem which I'm incorporating, but where do I go from there?

It has started me thinking on a very interesting set of questions: Can you use neuroevolution in a cumulative fashion so that you train a network to do one thing and then train it to do something more advanced based on the initial training? It's the adage that you have to crawl before you can walk, and walk before you can run (even though one isn't use to do the next).

Are there learning tracks where you train a neural network the first thing on the track and that prepares it to learn the next thing, and so on? Or is it more of a network where each learning outcome is a node and the nodes feeding into the next layer of nodes all need to be learned before the next layer can be learned? This seems more logical to me. Not only does a person need to learn how to crawl, but they also must learn how to stand before they learn how to walk. Maybe a feed-forward network, since it's unlikely that something learned down the line would be required for an earlier learning node (though back-propagation does seem in effect to strengthen the earlier understandings).

The question then becomes, what starts the whole thing off? What are the first things that babies learn and must master before they can progress to the next stage of learning? Certainly as a person gains experience in multiple, cross-cutting fields (e.g. music, language), the inspiration and understanding to progress becomes more complex, but what about at the very beginning? Is it something that can be mapped out? Has someone already done this?

Where Did Summer Go?

Or even Autumn for that matter?  One week it's summer and the next it feels like we skipped directly to Winter.  The picture to the left is from our porch in Castle Rock, CO where it has started snowing.  We had snow a week or two ago too, but this is the first time it's actually stayed on the ground.  I didn't think this was supposed to happen at least until Halloween!

I suppose it was a sign of the season as Erin and I went to get our flu shots on Tuesday last week.  While our arms both felt sore afterward, we weren't otherwise side-affected.

Still, when my friend Andres invited me to watch baseball last Thursday I thought it was a good omen for the last remnants of Summer.  While the game was a lot of fun and the Rockies won, I was sorely disappointed in terms of the weather.  It was okay in the sun, but the wind was bitterly cold in the shade where we were seated.  It was nice to get after the game and get a beer and a burger.

Then this past weekend Erin and I attempted to ride up to see the Fall foliage like we did last year (Enjoying Fall Foliage, Ouzo, and Bowling).  This time we were able to go with our friends Brian and Cassie (also on their motorcycle).  After meeting our wedding videographer to see a first cut of our wedding video (which looked very nice) we met up and headed up to the mountains.  The ride started nicely, but got colder the higher up we got until we stopped at a light and Erin exclaimed how cold she was.  Still, I pressed on hoping to find a good spot to view the changing colors.  After a while though I realized that it wasn't going to happen with the route we had chosen and the approaching storm clouds.  We turned around and went back to Brian and Cassie's place.

Back in the warmth of their home we were treated to a warm, delicious meal and a touching movie.  We also got to spend time with Neal's dog Bart (pictured right) who Brian and Cassie were looking after for the weekend.  He's such a friendly and well-trained dog that it's hard not to like him.

On Sunday Erin and I did some work and ran errands until it was time to start preparing dinner.  It was our pleasure to host my aunt and uncle down to our place for some lemon chicken picatta.  We had a great time chatting and catching up.

So now Erin and I are getting ready to head out to our monthly supper club up in Denver.  Of course it would be snowing and I hope it lets up soon.  Fortunately Erin doesn't have work tomorrow so we can sleep in, stay inside, and hopefully keep warm!

Wisconsin

This past weekend Erin and I flew out to Milwaukee to celebrate one of her cousin's getting married.  As I had never been to Wisconsin before I was interested to see what it would be like.  So on Friday morning we got up early and got to the airport with no problems.  Arriving in Milwaukee in the afternoon we met up with Erin's parents who had rented a car.  Driving through the city on our way out to the city of Brown Deer where our hotel was provided a nice view of the city.  Much of the countryside reminded me of Georgia for some reason with its dense woods and lush green lawns.

At our hotel we met up with Erin's brother, Andrew, and his girlfriend, Sara.  Once we had all checked in we found that the hotel bar was serving martinis and decided to have a drink.  After finishing our drinks we thought it would be good if we went out in search of a walk to do before the rehearsal dinner.  While we were able to find some nice river-side roads near our hotel it soon began rain and walking was out of the question.  So we returned to the hotel bar for more to drink as we played cards and waited for dinner.  The dinner was a delicious Italian meal out at the brides' parent's house where I got to meet much of Erin's extended family.  They were all very nice and we had a great time.  We even had a chance to sit around the fire pit as the rain let up.

On Saturday we got up and had breakfast at a nearby Denny's before heading out to walk around a lake we had driven by the day before.  The area was very nice and green, but with the sky still overcast and the humidity starting to be felt more heavily, it seemed to be a very wet area.  After getting cleaned up we were off to the traditional Catholic wedding just outside of historic Cedarsburg.  Afterwards, as pictures were being taken we had time to wander around the charming city before the reception.  One of our most favorite places was a brew pub down on the river.  In a very relaxing setting Erin finally found a beer that she liked (Framboise) and we all enjoyed the chance to talk.  As the reception was being held at the hotel we were staying at we eventually made our way back there for the festivities.  The new couple looked very happy together as they danced and we got a chance to see a slide-show of pictures from their lives.

For Sunday we had talked about several potential things to do before we all left.  We started out with some geocaching, using a GPS to find a box hidden by others.  While it took us a while to find the cache it turned out to be in a very pretty nature preserve.  We were then on our way back to city to see the Andy Warhol exhibit in the Milwaukee Art Museum.  The museum had some very nice exhibits and some exciting architecture.  It includes an elaborate, wing-like brise soleil which shades the museum (pictured as its opening).

Before too long though it was time to catch our flight.  We said our goodbyes and made out way to the airport.  It was good visiting with all of Erin's relatives and getting to see Wisconsin.  I certainly have a higher opinion of the area now that I've had a chance to see it first-hand.

Kart Racing, IMAX, and Pottery Painting

This past weekend was an exciting one for Erin and I as we did a diversity of activities.  Friday, after Erin got off work, we went over to the Union for happy hour - which is turning out to be a low-cost ritual for our Friday afternoons.  That evening we played a board game while eating apple pie and drinking port.  It was a very enjoyable experience that Erin had set up and shared with me.

On Saturday we went to celebrate our friend Andres' birthday out at the Centennial Track.  There we all got suited up with helmets, gloves, and overalls before climbing into karts for some racing around the .8 mile track.  While we only got to race for about 10 minutes it was fun.  After the race we hung out a bit with Andres and the rest of his friends before going over to Erin's parent's house for dinner.

Erin's sister Amy was in town preparing to defend her thesis on Monday, so we went to see her and visit with Erin's parents.  They took us all out to this authentic Mexican restaurant called "El Paraiso" in North-West Denver.  Unlike the typical tacos and burritos this place had a wide variety of traditional Mexican dishes.  The food turned out to be good and with generous portions.  There was also a mariachi band playing adding to the atmosphere.

More fun came later in the weekend as Erin and I went downtown to the Denver Museum of Nature and Science to watch an IMAX show.  Being located in city park, the museum's location gave us a chance to have a nice walk around the park before our show started.  We watched a show called "Wild Ocean" about an aquatic feeding frenzy that happens off the coast of South Africa each year.  It was amazing being immersed in the underwater action the way that an IMAX movie provides.

After the show we went to Ceramics in the City where we got to pick out a piece of pottery to paint.  After a long deliberation we settled on salt and pepper shakers along with a common theme of nature.  We both had a really good time as we painted our designs on our shakers.  We finally finished and gave the shakers to the attendant to be glazed and fired.  We're looking forward to seeing how they turn out next week.

Day to Day Stuff

Erin and I often joke about how the main reason for this blog is so that when we're old and our memory goes it will be the only way for us to remember what happened.  The thing is, sometimes I even forget to write and then can't remember what we did even two weeks ago.  So here's my best stab at what we've been up to (mainly gathered from what's still on my calendar):

The last week of August found me in the doctor's office for a regular checkup with everything checking out well.  Later in the week Erin and I met with an accountant to see if we needed his help with taxes now that we're married.  While he was very informative (and talkative) I don't think we actually make enough money right now to need his services (or afford them).  That weekend Erin and I had a nice day downtown as we went to the Tattered Cover and enjoyed a glass of wine in the Highlands.  That weekend also gave us an opportunity to get out on the motorcycle and ride around the countryside.

Last weekend Erin and her girlfriends went up to Vail for a jazz festival.  She really enjoyed seeing the Black Eyed Peas and being with her friends.  While I spent most of that time at home I went out several nights with my friends as well.  It was fun to spend time with our friends and on Sunday when Erin got back it was nice to be back together.

For this past week I've been dog sitting for my aunt and uncle.  They have a very nice dog that has a lot of energy and goes crazy sometimes around other dogs.  It's been good going up to see her each day and take her out for walks.  Since I only see her in the afternoons it has given Erin and I the opportunity to go out each night this weekend.  On Saturday we went out to watch a movie ("The Proposal") and have dinner with our friend's Brian and Cassie.  This night we went over to Erin's parents for a wonderful Lemon Chicken Picatta dinner.

So that's the last few weeks in a nutshell.  Hopefully I'll remember to keep up with the blog so it doesn't go so long.

Botanical Gardens Concert

Last weekend was busy as Erin and I were seeing people every night.  On Friday night we met a colleague of mine from a previous job.  He treated us to dinner at a local Mexican food place since he lives just a few miles south of us and wanted to congratulate us for getting married.  I really enjoyed the food and it was nice to catch up.  I think it will be nice to go down to Monument and see him and his girlfriend for dinner again when we get the chance.

On Saturday we had planned to have our friends Brian, Cassie, and Neil down for dinner as they have had us over for dinner so many times.  It took some time to prepare all the food and make sure that the place was all cleaned up, but we all had a great time.  I hope we can do it again in a month or two and maybe even make a habit out of it.  I think I was finally getting the hang of the Balderdash game we were playing and want to have another shot at it.

On Sunday, Erin's parents treated us to a concert at Denver's Botanical Garden.  All summer long the Garden has had bands playing on weekend evenings while people come out to picnic and listen.  They've also had an exhibit theme of dinosaurs throughout the Garden.  For our picnic/concert we listened to Los Lonely Boys and Los Lobos.  While it was very crowded as people tried to find good spots on the lawn around the arena, we were able to find a level spot that wasn't too far away.  It was very fun to have our picnic, talk, and listen to the music under the warm sunset sky.  In between sets we even got a chance to roam the garden and admire the plants and exhibits.

All in all it was a very social weekend.  It was a fun time, but now this weekend we're taking a much more relaxed pace.  I'm sure it won't be long though when we're out eating with someone else.