ADT: SmallVector size/capacity use word-size integers when elements are small
SmallVector currently uses 32bit integers for size and capacity to reduce sizeof(SmallVector). This limits the number of elements to UINT32_MAX. For a SmallVector<char>, this limits the SmallVector size to only 4GB. Buffering bitcode output uses SmallVector<char>, but needs >4GB output. This changes SmallVector size and capacity to conditionally use word-size integers if the element type is small (<4 bytes). For larger elements types, the vector size can reach ~16GB with 32bit size. Making this conditional on the element type provides both the smaller sizeof(SmallVector) for larger types which are unlikely to grow so large, and supports larger capacities for smaller element types. This change also includes a fix for the bug where a SmallVector with 32bit size has reached UINT32_MAX elements, and cannot provide guaranteed growth. Context: // Double the size of the allocated memory, guaranteeing space for at // least one more element or MinSize if specified. void grow(size_t MinSize = 0) { this->grow_pod(MinSize, sizeof(T)); } void push_back(const T &Elt) { if (LLVM_UNLIKELY(this->size() >= this->capacity())) this->grow(); memcpy(reinterpret_cast<void *>(this->end()), &Elt, sizeof(T)); this->set_size(this->size() + 1); } When grow is called in push_back() without a MinSize specified, this is relying on the guarantee of space for at least one more element. There is an edge case bug where the SmallVector is already at its maximum size and push_back() calls grow() with default MinSize of zero. Grow is unable to provide space for one more element, but push_back() assumes the additional element it will be available. This can result in silent memory corruption, as this->end() will be an invalid pointer and the program may continue executing. An alternative to this fix would be to remove the default argument from grow(), which would mean several changing grow() to grow(this->size()+1) in several places. No test case added because it would require allocating a large ammount. Differential Revision: https://reviews.llvm.org/D77621
Loading
Please sign in to comment