The Curious Case of Punk 4513
As noted in our overall project description, we consider our model at this stage to be still in 'beta'. Machine learning models do not work 'out of the box' in most cases - they still require significant tuning to improve their performance. Our current model is quite reasonable, but has some obvious areas for improvement - for example, it does not sufficiently penalize "floor punks."
Beyond the above, different machine learning models with different statistical approaches have their own biases. Some of our models have been working on have created too much dispersion of values; others have created too much clustering of values, particularly the tree-based models. In the case of punks, this sometimes has to do with how much the model is valuing the type (alien, ape, zombie, human) and how much the model is valuing the attributes (e.g. beanies, hoodies and so on)
In the 'debate' among our machine learning models, the single most controversial punk to-date was Punk 4513. Punk 4513 is obviously a valuable and unique punk by any standard. It is a rare Zombie punk with two quite desirable attributes - the Beanie Cap, probably the most desirable attribute from a valuation perspective according to our models, and the Luxurious Beard.
All the models agreed that Punk 4513 was valuable, with the lowest price prediction of any individual model being 300ETH (the last sale price for Punk 4513 was 185ETH) and most models clustering around 432ETH. In fact all models but 1 had Punk 4513 as the most valuable Zombie, but less valuable than the least valuable Ape. One type of model, however, was a huge outlier, valuing Punk 4513 in the thousands of ETH, regardless of parameters, and placing Punk 4513 within the apes or even the aliens in some cases (in effect giving a lot more relative value to attributes).
The final blended model has Punk 4513 as the most valuable Zombie by far, with a predicted value 558ETH vs 339ETH for the next most valuable Zombie Punk 6297 but just below the least valuable Ape, Punk 8219 at 660ETH.
Is this right? We are not sure. Our qualitative sense is that perhaps the model is overvaluing this Punk, but perhaps the model is seeing underlying patterns that we are not. We spent a lot of time looking at Punk 4513, but we are not willing to adjust the model, in effect manually, on behalf of one Punk, so for now he stands alone among the Zombies.
In time, the model will coalesce and come closer and close to reality and Punk 4513 might adjust up or down.
Our objective remains the same, to build objective, robust, defensible automated valuation models that can read out their implicit parameters and assumptions and expected variance or error rate. We think this is an endlessly fascinating challenge at the intersection of crypto-assets, art and machine learning and look forward to the journey.
We welcome all and any feedback from the wonderful CryptoPunk community.
https://nftvaluations.com/cryptopunks/4513